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"PROGRAM MODELS FOR EMR STUDENTS" 
OHIO ESEA TITLE III PROJECT #45-71-207-2 

A project Involving eight regional sattellte centers and a Stato-wido coordinating 
center was created to implement objectives meeting the foUoving needs i 1. create 
behavioral objectives for the EMR curriculum, 2. train teachers of EMR students 
to write and field test the bohavioral objectives, 3. develop a system to evalu- 
ate the etfect of various program models on the behavior of EMR students. 

During the two years of the project hundreds of EMR personnel in urban, suburban, 
and rural school districts across the State were involved in activities to meet 
these needs. Objectives were developed In fourteen curriculum areas and form 
Che basis for a curriculuui. Eight models were created and 99 different classes 
of studentj were initiated. A test was created which represents Items in a7.1 
fourteen curriculum areas and was given to all students in thece model classrooms. 
An evaluation system was created and is being used. 

Conclusions and recommendations Indicate a need to continue to refine the objccfc- 
Ives into a major curriculum for EMR students, to develop the teacher training 
component utilizing not only the regional staff but also nine university staffs, 
and to continue to Initiate more model classes and evaluate student progress. 
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B. CONTENT DESCRIPTION 
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B COHTEXT DESCRIPTIOK 



Programs for educable mentally retarded (BIR) students' have had a long history in 
the State of Ohio. Individual school districts have sponsored programs since the 
1890's when Cleveland Public Schools initiated the first special classes. In 
19'4l4-'+5» the Ohio Legislature took action to place the responsibility of a State- 
wide program vtth the Ohio Department of Education* Since that timei nearly 2I4 
years a^o, a groat number of students have progressed through an increasingly 
larger number of classes. In 196O-6I, 20? school districts employed 855 teachers 
to provide instruction for over 13,000 students. Ao a result of the largest 
expansion in its history, Ohio opens the 1970-71 school year with 1^98 school 
districts employing 3,192 teachers of k6y9l5 educable mentally retarded children. 
Incidence figures indicate that this number is approximately C3 percent of all 
students who could qualify for the progroja. 

There were three needs within the EMR program toward which the efforts of this 
project focun. First, there was a need to develop a method of evaluating the 
effectiveness of EMR programs. Second, there was a need to evaluate existing 
models of El-IR programs to determine which are most effective* Third, there was 
a need to develop a process of implementing the results of this project with 
teachers of EMR students. 

Need to Develop Method of Evaluation 

Research studies conducted in Ohio (Crim, 1969) indicate that 80 percent of the 
students who are successful in the EMR curriculvun are maintaining good employment 
record as adults. Contrary to this, 70 percent of the EMR children who drop out 
of school at 16 years of age or before become dependent on state and local tax 
funds in the form of welfare payments or institutional costs. 

The effectiveness of these programs, however, have been challenged by other 
criteria than that of this research in Ohio. 

In an attempt to determine the most appropriate educational program model for the 
EMR, a number of research studies have been undertaken since 1932. The primary 
approach utilized has been a comparison of EI^ children placed in special classes 
with EMR children remaining in regular classes. 

An examination of these studies indicate Inconclusive findings: Dennett (1932), 
Pertsch (1936), Johnson (1950), Cassidy and Stanton (1959) report that the 
children left in the regular grades are, on the whole, superior academically to 
the children assigned to special classes. 

Regarding social acceptance and social adjustment, Johnson (1950-1962), Baldwin 
(1958), Jordan (1959), indicate that the regular class does not provide a social 
climate conducive to healthy emotional development, and Cassidy and Stanton (1959), 
Goldstein (196I4) indicate that the special class group showed superior social 
adjustment. 

Still other studies Wrightstone et, el, (1959), Ainsworth (1959) indicate no 
signif leant differences mong the groups in academic achievement. 

There is presently no adequate criteria against which programs can be compared. 
Research over the past ho year^ has used academic achievement, social acceptance, 
and liQi gains as criteria to evaluate EMR success. Commenting on past research. 
Kirk (I96U) indicates that all of tne completed studies suffer froin the problems 
iP. 8|tu investigation. 



None controlled the essential variables needed for adequate inference. One of the 
weaknesses cited by Kirk is the poor reliability and validity of the aeasurenent 
instruments used in comparative studies • This lack of ''effectiveness criteria" 
is a basic source of contradictory and equivocal research of efficacy of El"® 
classes . 

Current educational emphasis on the dovelofjaent of specific behavioral objectives 
day offer the most significant advancement in educational evaluation in recent 
decades. 

Emphasizing this need, Bloora (1956) points out, "It should be clear that objectives 
are not only the goals toward which the curriculum is shaped and toward which 
Instruction is guided, but they ar$ also the goals that provide the detailed 
specification for the construction and use of evaluation techniques." (P* 27) • 
Also, Mager (1962) states, "When clearly defined goals are lacking, it is 
impossible to evaluate a course or program efficiently." (P. 3) 

Meyen (1968) suf^gests that the vagueness of educational objectives for the educable 
mentally retarded forces teachers to resort to personalized interpretation which 
may result in unwarranted emphasis being placed on skills and concepts which may be 
inappropriate. The vagueness of these educational objectives make assessing 
achievements of the EMR an almost Jmposalble task. 

In view of the past researoh, It is clear that one of the major obstacles preventing 
an effective evaluation Oi the educable mentally retarded program has been the lack 
of specific measurable objectives to be used as evaluation criteria. Therefore, 
in order to determine the most appropriate educational program, the development 
of these objectives was essential* T he re was a need to develop a series o f 
measurable behavioral objectives to be used by educators as evaluation criteria for 
Judging the effectiveness of EMR classes . 

Need to Evaluate Existing; EMR Models 

The Ohio Division of Special Education program has been growing dramatically in 
the last decade and has been somewhat successful (based on follow-up data) as 
noted previously. However, it was apparent with continuing increases in units for 
EMR students and with population projections, that the self-contained classroom 
modv^l would not be efficient In the future. Consideration of this matter took 
into account the teacher shortage, the classroom shortage, the cost per pupil 
factor, and the crisis in educational financing. In addition to this, is a rising 
national movement to completely restructure the self-contained classroom model as 
it now exists (Dunn, 196^^). The basic argument is that the self-contained class 
is, in fact, an unconstitutional segregation of a group of students Into a tracked 
educational program. The issue has received several court hearings including the 
Judge J. Skelly Wright decision in the District of Columbia in 1967. Judge Wright 
ordered that the tracking system be abolished, an action which dumped hundreds of 
EMR students back into regular education classes overnight. It vas Imperative 
that an evaluation of existing models of educating EMR stu de nts be conducted t o 
provide direction for future programs . It was anticipated that in the seoond 
and third phases of this proposed project, several different educational EMR 
models would be evaluated on the basis of the behavioral objectivf? criteria arid 
other identifiable factors to determine the most effective methods of educating 
these students. 
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Need to Implep ent Res ults of This Pro.)ect 



The third statement of 'V^eodi out of vVilch this project was designed, is ceritored 
around the problem of disseraination of research results and the implementation of 
the conclusions at the local school district level. The common cry of educators 
for decades and the common criticism of educational research has been the fact 
that research results are either extremely slow in grass-roots effectiveness or 
without impact on the local level. There wa s a need to develop a process of change 
in the educational milieu which wo u ld effectively change the behavior of teachers 
and st udents as Quickly as research evidence^ is available . This project attempted 
to meet this need by working with a substantial number of teachers throughout the 
project in an effort to change their behavior as research results indicate. 

To carry out the objectives of this project, eight satellite centers were established 
in Ohio. Figure 1 indicates the regions which were covered by those satellites % 



Figure 1 
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The Coordinating Center, located in Mentor, Ohio, provided direction for the 
implementation of the activities. Chart i shove the staff and financial arrangements 
for the satellites. 

An Advisory Board was established to oversee and direct the project. This Board 
consisted of local representatives from each region, the satellite coordinators, 
representatives from the Ohio Division of Special Kducation and the Ohio ESBA 
Title in office. Chart i also shows the composition of that Board. Each satellite 
area carried out the intent of the project in their local area and reported back 
to the Coordinating Center and Advisory Board. 
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T 
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dJ..UJ.LU 

Each of the 
Satellite Centers 
will have a rep- 
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Board and the Project 
Consultant on this 
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C. PROGRAM EXPLANATION 
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C PROOMM BXPIAMTION 



Total federal support under ESEA Title III 

Total federal support othor than under ESEA Title III 

Total nonfederal support 

Total project cost 

Total evaluation cost 

Local E d ucation Asencies Served by the Pro.1eot « 



Akron City 



Ashtabula 



SCHOOL 
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31 
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11 
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10 


Goodyear Junior High 
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25 


Duchtel High 
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18 
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1J< 


West Junior High 
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18 


Geneva Elementary 


652 


16 


Grand Valley Elementary 


600 


26 



ERIC 



SCHOOL 



TOTAL 
ENROLIMENT 



Ashtabula 
continued 



Athens 



TOTAL DIRECT 

STUDENT 
PARTICIPATION 



Jefferson High 


1*78 


16 


Lakoview High 


1800 


16 


Liberty High 
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17 


North Elementary 


U65 


as 


MapXewood East 


U2i6 


sd 
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Pymatuming ■ Valley High 
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Maplevood High School 
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39 


Newton Palls High 
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Champion Center Elementary 
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16 
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600 


17 


l8t Street Elementary 
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Poland Senior 


679 


16 


Springfield Elementary 


590 


!»♦ 


New Springfield Elementary 


25h 
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East High 


1275 


16 


Washington Elementary 


5I46 


12 


Rayen High 


1221 


32 


Lordstown Elementary 


395 


12 


Market High 




US 


Matthews 


577 


19 


East Junior High 


708 


lil 


Conneaut High 


786 


18 


Geneva Junior High 


682 


28 


pymatuming Junior High 


133 


15 


Bazotta Elementary 


1*30 


ll» 


Austenburg 


1*77 


16 


Williamfleld Elementary 


189 


ll» 


Chestnut Elementary 


I4I2 


15 


Berlin Center Elementary 




13 


Maple Ridge Elementary ^ 


-> 366 


ll* 


John Davis Elementary 
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12 


Lincoln Middle 


681* 


17 
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ll» 


Jefferson 




111 


Bascom Elementary 
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18 
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15 
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15 
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30 
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3129 


15 


Alexander Local 


1555 


lU 


Athen City 


3856 


15 


Federal Hocking Local 


1720 


13 
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TOTAL DinECT 



SCHOOL TOTAL STUDENT 

NAME ENROLUnSNT PAHTlClPATION 



Athens .NelsonvUlo York City 2115 1*2 

continued Meigs Local 3060 30 

Oallipolis City 22^1 1? 

North Gallia Local 965 15 

Port Frye Local 17^8 9 

Marietta City 55l6 30 

Switzerland of Ohio 1*956 Ik 

Caldwell Exempted 1505 10 

Hamilton Ach Junior High 756 19 

Columbian Elementary 525 15 

Oreenhills Middle 920 13 

Harrison Elementary 986 12 

Heinold Junior High 1057 20 

Lincoln Heights Elementary 776 12 

Lockland Middle 653 9 

Merry Junior High 755 20 

North College Hill High 1050 16 

Robert E. Lucas Int- 756 37 

Sawyer Junior High 10^5 i?U 

Sherwood Elementary 9OO ?3 
Terrace Park Elementary 3^0 

Washington Elementary 778 15 

Withrow High 2599 10 

Harding Junior High IO2I4 13 

New Miami Middle 1*30 li> 

Roosevelt Middle 786 1(\ 

Somerville Elementary 195 1? 

Berry Middle 1250 17 

Francis Dunlavy khO 11 

Hampton-Bennett Elementary 5OO 15 

Pennyroyal Elementary 38O 29 

Batavia High 320 16 

Cook Elementary 85O 23 

Clenaont Northeastern High 85O 29 

Goshim Primary 1100 12 

Merwin Elementary 670 lU 

Milford High 1125 12 

Clarksville Elementary 625 8 

Cedarville High 1*03 21 

West Main Elementary 68O 8 

Jefferson High 575 23 

Page Manor Elementary 775 ih 

Louisville Louisville High 1020 15 

Trump Road Elementary 305 12 

North Industry Elementary 397 15 

Marlln^ton Middle 900 16 

Edison Junior High 1610 ik 

Jackson Elementary 81O 10 

McKinley Elementary 58l 15 
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TOTAL DIRECT 



SCHOOL tooal student 

JAl^ ENR0I;LMENT PARTXCIPATION 

Louisville MoKlniey High 2038 l8 

continued Pleasant Orove Elementary kOO 12 

Reilly Elementary J'tO 13 

Horace Mann Elementary 89 13 

Hoover High 1200 12 

Morgan Elementary l»26 12 

East Liverpool High 1350 l8 

North Street School 530 12 

Walker Junior High 550 18 

Middlebranch Juiiior High 950 15 

Hazel Harvey Elementary 758 12 

Lincoln Elementary 55'* 1^ 

North Street School 530 13 

T. C. Knapp Elementary 635 15 

Lura B. Kean Elementary 150 12 

Summit School 926 12 

Orrvllle High ih^ 22 

Mary Irene Day 960 lU 

Jackson Elementary 8l8 12 

Sandy Valley High 1050 l8 

Reilly Elementary 5^0 12 

Marllngton Middle 900 lU 

Sauder Junior High 1075 l6 

Lake Middle 800 13 

Lake Middle 800 12 

East Sparta 21*5 15 

Waynesburg 275 l6 

Davsett Elementary 550 13 

Dawsett Elementary 550 12 

Osborn Elementary k53 l6 

New Albany Elementary 525 10 

Brown Elementary 195 ' 12 

Brown Elementary 195 12 

Avondalo Elementary 520 12 

Mentor North High - Willoughby-Eastlake 2175 ll* 

North High - Willoughby-Eastlake 2175 12 

South High - Willoughby-Eastloke 11*00 15 

South High - Willoughby-Eastlake lltOO 11 

Willowlck Junior High 2300 22 

Willowick Junior High 2300 12 

McKinley Ulh 13 

Royalview 979 12 

Shoregate 1*95 11 

Shoregate h95 12 

Claridon Elementary - Berkshire 228 15 

Mlddlefield Elementary - Cardinal 587 11 

Kens ton Middle School - Kens ton 633 19 

Chester Elementary - West Geauga 5^3 8 

Chester Elementary - West Geauga 5''*3 \k 

Mentor High 2608 lU 
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TOTAL DIRECT 



SCHOOL TOTAL STUDENT 

JJAMB_ ENROLLMENT PARTICIPATION 

Mentor Shoro Junior High 1193 18 

continued Shore Junior High 1193 13 

Shore Junior High 1193 12 

Bellflover Elementary 12 

Fairfax Elementary U 

Hopkins Elementary 1*71 12 

Madison High 1157 32 

Memorial Middle School, Madison 3^0 Ik 

Memorial Middle School, Madison ^Uo 19 

Harvey High School - Palnesvllle 1070 13 

State Street Elementary - Paines- 320 15 
ville 

Hale Road - Painesville Twp 1*00 15 

Wickliffe High 1000 19 

Wickliffe Junior High 1100 13 

Mapledale Elementary i*6o 9 

Worden Elementary 900 15 

Bryden Elementary - Beachvood 309 11 

Victory Park - South Euclid 367 13 

Victory Park - South Euclid 367 11 

Eastvood Elementttry - Warrenaville 1*75 13 

Chagrin Falls Middle 855 12 

Roosevelt School - Euclid 6l7 12 

Shore Junior High - Euclid 1139 I'* 

Euclid High 29U9 25 

West Junior High - Maple Heights 78O 22 

Maple Heights High 160O I8 

Belvoir Elementary, Cleveland Hts. 51*6 12 

Mayfield High 2000 12 

Mayfield High 2000 11 

Shaker Heights High 1820 Ik 

Toledo Harvard 722 33 

Sherman 10l*8 55 

Reyraor . IO63 53 

Waldbridge 879 26 

Hamilton 10l*l 36 

Fall-Meyer 391 26 

LaOrange J2k 32 

Westfield 553 I6 

Whittler 1501* 1*0 

Jones 780 19 

Riverside 825 16 

Start High 2175 ik 

Woodward High 2510 20 

Rogers High 2300 13 

Waite High 2075 15 

Nronac Elementary 521 15 

Whitmer 2730 23 

Jefferson Junior High 921 18 

Hiawatha Elementary 5^6 16 

Lake Elementary 810 13 
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TOTAL DIRECT 

SCHOOL TOTAL STUDENT 

.HAt€^ EHROLLMEUT PARTICIPATION 



Toledo Stranahan Elcfinentary 800 31 

continued Sylvonla High 2600 23 

Jackson Middle 265 17 

MoCoab Elementary 256 13 

Delta Junior High 300 13 

Otis Elementary 1*91 37 

Slocomb Elementary 365 ih 

Defiance Junior High 1C60 19 

Wynh Elementary 3T5 17 

Clyde High 850 17 

Holland Elementary 725 32 

Maumee High 1I425 11 

Antwerp High 532 I8 

Central Elementary 832 15 

Napoloan Junior High 1100 17 

Kalida Elementary 590 12 

Fallen Timbers 8OO 13 

Shervood Elementary 36O 13 

Crestviev High 366 9 

Hopewell Lar^.on High 1|86 13 

Tontogany Iiiiermediate 235 12 

North Central Junior High 320 11 

Ottoville Elementary 550 13 

Seneca East High U50 8 

Wayne Trace Elementary 700 17 

Rocky Ridge 213 13 

Ohio City Liberty Elementary 300 12 

Allen Central Elementary 500 ih 

Hilltop High 250 13 

Woodmore Intermediate 510 12 

Luckey Elementary 260 17 

Van Lue High 229 1** 

Central Junior High 950 38 

Findlay High 2110 20 

Whittier Elementary 719 23 

Tuscarawas Wintersvllle Elementary 720 12 

Millersburg Elementary 61O 15 

Tuscarawas Elementary 679 15 

John E. Gregg Elementary 728 ll* 

Pleasant Hill Elementary hl2 10 

Nashville Elementary 230 12 

Hills Elementary I169 12 

Brilliant Elementary li31 1** 

Lakeville Elementary 175 1^ 

Bolivar Elementary I+70 16 

Baltic Junior High 395 13 

Maysville Junior High 6T0 20 

West Muskingum High 88O 12 

Newcomerstown High 500 16 



ERIC 



14 



TOTAL DIRECT 



SCHOOL TOTAL STUDENT 

JIAME_ ENROLUdEUT PA RTICIPATION 

Tuscarawas Indian Velley South High 368 10 

continued Rivervlew High 985 19 

Oronvillo Middle 1*00 10 

Watkino High 1*10 13 

Hartaler Elementary 3^6 12 

Buffalo Elementary 2l*0 13 

Mc'lvern Elementary 1*25 15 

Toronto High 675 10 

St. Clairsville Middle 9(57 I6 

Carrollton Junior High 1200 17 

Walnut Creek Elementary 177 12 

Licking View Elementary 260 13 

Tuscarawas Valley High 807 I6 

Claymont Junior 6k3 17 
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SCHOOL TOTAL TOTAL DIRECT 

.Mm.. E NROLLMEN T STUDENT 

Clovoland Miles Standlsh 875 31 

Miles Park 670 16 

Milos 1000 31 

Doan 525 24 

A, J. Riokoff 875 32 

Wooldrldge 270 15 

Woodland Hills 775 15 

Anton Grdina 900 29 

Stanavd 380 39 

Stetling 600 18 

Milliam H. Brett 450 15 

Tremc'.it 1125. 28 

Pavkvood 650 15 

Waverly 625 16 

Orchard 1100 29 

Dunham 800 16 

Clark 500 15 

Captain Roth 1025 18 

H. W. Longfellow 485 31 

J. W. Raper 825 16 

Mount Auburn 570 19 

M. B. Martin 700 16 

Hough 625 16 

Marion 390 28 

J. Burroughs 275 17 

J. F. Landis 775 16 

Beehive 850 18 

Harvey Rice 690 18 

Central 1190 27 

Thomas Jefferson 1360 15 

Glenville Senior High 3150 20 

Nathan Hale 1600 14 

John Hay 1900 17 

Rawlings 1050 15 

John Adams Senior High 2800 33 

L. Diehl Junior High 760 14 

N. Baker 690 15 

Lincoln 1535 32 

• John Marshall 2850 13 

Empire 855 29 

South High 1900 15 

Wilson Junior High 765 17 

East 1850 39 

Kennedy 2900 15 

West Junior High 1570 17 

West Tech 1200 18 

Spellacy 1010 34 

Franklin D, Roosevelt 1450 16 
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SCHOOL TOTAL 
^IAME ENROLLMENT 



Columbus City Kubbord Elementary 442 

SuUlvant Elemontary 439 

Linden Elementary 1709 

Dona Avenue Elementary 601 

Wolnlacd Park 487 

Felton Elementary 254 

East Linden Elementary 547 

Siebert Street 4A4 

South MlffUn 741 

Eastwood Elementary 219 

Starling Junior High 933 

Starling Junior High 933 

Franklin Junior High 831 

Franklin Junior High 831 

Everett Junior High 925 

Medina Junior High 1141 

Champion Junior High 723 

North High 1380 

North High 1380 

Duxberry Elementary 588 

Medary Elementary 501 

Everett Junior High 925 

Champion Junior High 723 

Dayton City Ruskin Junior High 758 

Stelvere High 1244 

Jefferson Elementary 864 

Jackson Elementary 696 

Whittler Middle 707 

Van Cleve Elementary 712 

Dunbar High 1323 

Orrville Wright Middle 932 

Cleveland Elementary 1238 

Cornell Heights Middle 738 

Emerson Elementary 722 

Longfellow Middle 946 

McFarlane Middle 880 

McGuffey Elementary 720 

Weaver Elementary 809 



TOTAL DIRECT 

STUDENT 
FARTICIP ATION 

20 
U 
16 
16 
16 
12 
16 
16 
12 
IS 
16 
15 
17 
16 
19 
16 
18 
15 
20 
28 
16 
19 
13 

22 
51 
65 
13 
9 
11 
12 
34 
21 
22 
13 
37 
A8 
15 
11 
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D. EVALUATION OF ACTIVITIES MID OUTCOMES 
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D EVALUATION OF TtFE ACTIVITXES AMD OUTCOMES 



This part of the narrative includes information from both years of the project. 
However, many times when data has been tabulated in the ^*Pirst Year Evaluation, 
1971-1972,*' the reader is referred to that doctimcnt. The goals ani objectives 
listed below are taken directly from the project proposal for the second year of 
the project* 

Goal ^1 V altdate a series of meaourable behavioral objectt vefi. 

Objecti ve lA Recruit 200 teachers of students across the participating 
geographical areas for validation of the ih sets of previously written behavioral 
objectives during February 1, 1972 - June, 1972. 

Objective IB Recruit l60 teachers of EMR students across the participating 
geographical areas for validation of the ih sots of previously written behavioral 
objectives during September 1, 1972 - January, 1973* 

This project} has involved, as is indicated in Table 1, over a thousand teachers. 
These teachers represent all geographical areas, (urban, suburban, and rural) in 
over 50 counties and over 200 school districts. Each of these teachers have a 
minimum of 10 hours of in-service education plus field testing of the behavioral 
objectives and as high as 30 hours of in-service education and many, many hours 
of field testing objectives over this two-year period. 

The field test process has been one that we have used to validate the behavioral 
objectives. The validation procedure has evolved in two basic areas; (l) the 
anecdotal sheets that teachers were using gave their anecdotal comments about the 
objectives as to the relevancy, and as to the applicability to the age level of 
the child. Copies of those field test anecdotal sheets can be found in Appendix A, 
(2) an accounting of student performance per objective. Student performance has 
been compiled on a mental age basis as to the general ability level of students 
passing and failing that objective. Sample copies of the results can be found in 
Appendix B, 

The entire first print out of this field test data is on file at the Mentor 
Coordinating Center. Sample sizes are relatively small per objective. That is, 
only a few number of students were given the pre/post field test process for each 
objective (range 0-90 students). In the year and one-half of field testing, 
81*5 different objectives have been recorded in the areas of reading, writing, 
and arithmetic. Hundreds of others were field tested for revision information 
but not recorded for information ^-aUtod to r^oatnl age and student cblllty. 

Table II lists the results of pre/post tests of teachers who were involved in 
the spring and fall sessions Indicating significant teacher gain as a result of 
the pre/post -^jesting of those training sessions. Copies of that test can be 
found in Appendix A. 



ERIC 



18 



TABLE I 



PROGRAM MODELS FOR EMR STUDENTS 
E8EA TITLE III 
TEACHER INVOLVEMENT 






opr 4 ng 
♦71 


PaI 1 

•71 , 


DuD" 

Total 


•72 


»72 


oil 0*» 

Total 


'71 - '72 


Akron City Schools 


S8 


33 


61 


hi 


ko 


81 


lk2 


Ashtabula County Schools 


28 


33 


61 


25 


28 


53 


111* 


Athens County Schools 


he 


ko 


86 


25 


20 


«i5 


131 


Hamilton County Schools 




57 


103 


29 


ll5 


7h 


177 


Louisville Locul Schools 




58 


102 


28 


32 


60 


162 


Mentor Exempted Village 


28 


33 


61 


kl 


ho 


81 


XU2 


Toledo City Schools 


ho 


39 


79 


25 


36 


61 


ll«0 


Tuscaravas County Schools 


29 


29 


58 


55 


Ik 


129 


187 


TOTAL 


289 


322 


611 


269 


315 


$8U 


1195 



Ma.lQr Cities (three ) 



Coliuabus City Schools 


51 


16 


67 




Cleveland City Schools 


26 


15 


Ul 




Dayton City Schools 


12 


15, 


27 






Sub-Total 89 


1*6 


132 


132 




Total 358 


361 


716 


1327* 



•Total figure includes duplication of those teachers vho participated throughout 
the project. 
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TABLE II 

SUfa-lARY SHEET PRE POST TESTING 
Fall 1972 

Models T flocheys 



Satellite 


& 






t 


Mentor 


8 


2,16 


8 


3.44** 


Trumbull 


6 


1.93 


6 


.79 


Akron 


13 


2,36* 


13 


5,09*** 


Tuscarawas 


10 


2.61* 


10 


3.67** 


Louisville 


11 


1.91 


11 


2.37* 


Hatnllton 


11 


1.47 


11 


3.44* 


Dayton 


12 


.83 


12 


1.02 



* Significant .05 
** Significant .01 
Significant .001 



Field Testing Teachers 

Part I Part II 



Satellite 


j\ 


t 


n 


t 


Toledo 


10 


1.12 


10 


2.39* 


Mentor 


14 


2.27* 


14 


6.09*** 


Trumbull 


14 


9 . 30*** 


14 


12,57*** 


Akron 


15 


2.19* 


IS 


7.55*** 


Tuscarawas 


4 


1.56 


4 


4.07* 


Louisville 


12 


.91 


12 


4,62*** 


Hamilton 


21 


3.48** 


21 


9.17*** 



* significant .05 Level 
** Significant .01 Level 
*** Significant ,001 Level 
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PRELIMINARY RESULTS OP PRE-POST TESTING 
OP 

OLD PROJECT TEACHERS AND NEW PROJECT TEACHERS 
Sprin8~-1972 

Old Project Teachers - Those Inyplved in spring and fall 1971 
New Project Teachers - Those teginnlng in February, 1972 
Pre-Test - February, 1972 
Post Test - May, 1972 



Pre Testing 

New Old . 


Part I 1 N«59 
of Test L *33.98 


Nsl30 1 ^ ^ »u 


Part II N«i*9 
of Test 9.71 


11.91 * ^'^^ 



*mean scores 

Teachers who previously had been Involved in the project scored significantly 
higher than those vho had not been involved* 

A « little or no awarene&s of behavioral objectives 
B « written and used in classrooia 





A 


B 


Part I 


Nt=36 


Na92 


of Test 


»33.50 


36.82 




A 


B 


Part II 


Na26 


N=82 


of Test 


9.01* 


12.08 



t « 2.23 



t * 2.9'+ 



*mean scores 

Teachers vho previously had knowledge of behavioral objectives scored significantly 
higher than those with little awareness. 

Major City Teachers - Cleveland 





?re-Test 


lost -Vest 


Part I 


N=i*0 


NsUO 


of Test 


*26.68 


31*. 80 








Part II 


N=i»0 


N=1»0 


of Test 




11.78 



t = I*. 51 



t « 6.97 ** 



•mean score 
»»signif p <.05 



•^^'"ihers vho went through the 10 hour training session including the field testing 
jip^edures scored significantly higher on the post-test than the pre-test. 



Dy, Edward Meyen, Project Consultant * has indicated both in his summary cornments 
after the first year of the project (see First Year Evaluation, 1971-1972} and in 
his comments in January > 1973 (see Appendix B) his observations of the success 
of this objective • 

The process used to achieve these objectives has been determined to be very 
effective. State-wide direction from the Coordinating Center in Mentor and 
local implementation at each local satellite is a viable way to carry out a 
project of this scope • The nation-wide validation team report (Appendix C) 
indicates a very successful achievement of this process. 

Goal II Evaluate EMR models in Ohio to. provide direction for future programs i 

Objective IIA Field test the behavioral objectives instrument on student 
behavior by June, 1972. 

The records of the completion of this objective can be found in the following 
report on the development of the Ohio Special Achievement Inventory (OSAl). 
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SECOND-YEAR REPORT 
on the 

OHIO SPECIAL ACHIEVEMENT 
INVENTORY (03AI) 
for 

EMR PROGRAM DEVELOPMSKT 
from 

BATTELLE CENTER FOR IMPROVED EDUCATION 
January 31 » 1$73 



The Mentor Public Schools awarded Battelle^s Columbus Laboratories a contract to 
provide technical advisory services In the area of computer data processings and 
research and development services toward the design of an experimental achievement 
Inventory to aosess the relative attaliunent of behavioral objectives by educable 
mentally retarded (EMR) students In the dlffer^;nt program models of their Title III 
project, "Program Models for EMR Students". 

The project effort was oriented t-oward two primary tasks: 

1) The first stage of the design of an achievement Inventory, consisting 
of items that would be representative of the behavioral objectives 
generated during the first year of the project, and 

2) The design and Implementation of a computerized data base for the evalu* 
atlon of EMR programs « 

Under the first task, Battelle-Columbus was responsible for the following workt 

1) Planning the content of each level of the achievement Inventory, 

2) Writing test items for each level, and 

3} Designing the first draft of each level of the Inventory « 
Under the second task, Battelle**Columbus was responsible for two areas of workt 

1) The design of a computerized data base, and 

2) The development of computer programs required for using the data base« 

This report presents a summary of Battelle-Columbus^ effort conducted under the 
first task» A separate report describes the effort directed toward the design 
and implementation of a computerized data base for the EMR programs evaluation. 

OSAI DEVELOPMENT CONTRIBUTORS 

Ta sk Leaders 

Cliff Audretch, Field Test Administration 

Ottolle Mills, Test Production and Second-Year Report 

John Powers, Management 

Allen Schenck, Data Analysis 

Mary Beth Zak, Data Processing 
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Asflooiates 



John Belland 
Don Gavin 

Ellculae Connolly- Co 11 ins 

Alan Cotzin 

Ted Doll 

Jin Eachbach 

Judy McCtacken 

Tom Noffaingar 

Bob Peauit 

Shirley Powers 

Marsha Rigano 

Bob Rubeck 

Roger Trent 



INTRODUCTION 



The Ohio Special Achievement Inventory represents a series of objective-based 
test items. The purpose of the achievement inventory Is to serve as an indicator 
of student progress in achieving the behavioral objectives of the EMR curriculum. 

The behavioral objectives were designed to guide the teacher of educable mentally 
retarded (EMR) students In selecting learning experiences and to aid her/him in 
evaluating student progress. The 14 categories of objoctivos wore adopted 
from the 14 poMlstlng life skills outlined in the Cincinnati EMR Curriculum 
Guide. ^ Prom the original 15,000 objectives written by teachers, the present 
draft of 7,000 objectives was developed. The 7,000 objectives represent the 
14 curriculum areas and 73 sub-curriculum areas within the curriculum areas* 
The curriculum areas include the following: 



to Appreciate, Create and Enjoy Beauty 
to be a Responsible Citizen 
to Communicate Through Arithmetic 
to Communicate Through Reading 
to Communicate Verbally and In Writing 
to Barn a Living 
Homemaking and Family Skills 
to be Healthy 
to Live Safely 
to Manage Honey 
to Travel and Move About 

to Understand Oneself and Get Along With Others 
to Understand Physical Environment 
to Use Leisure Time VJisely 

The objectives were written for each of four grade levels: K-3 or Primary, 4-6 
or Intermediate, 7-9 or Junior High, 10-12 or Senior High, 



1) 


teaming 


2) 


Learning 


3) 


Learning 


4) 


Learning 


5) 


Learning 


6) 


Learning 


7) 


Learning 


d) 


Learning 


9) 


Learning 


J.0) 


Learning 


11) 


Learning 


12) 


Learning 


13) 


Learning 


14) 


Learning 



"Behavioral Objectives for * Program Models for EMR Students''', A Title III 
Project directed by Thomas Noff singer, Mentor Exempted Village Board of 
Education, 1972. 



ERIC 



^ 24 



Thd objeoeivd of Battel U-*Columbud LaboratorleSi having baen contracted by 
M^ntot Public Schools > was to design ai\ instrument that would be an indicator of 
overall atudont achieve^ment of the behavioral objactives. The inventory should 
serve as an indicator of a student 'a achievement in each of the 14 curriculum 
areas. Since four grade levelo of EMR instruction exist, a separate version of 
the inventory was needed to evaluate each grade levels Primary, Intermediate, 
Junior High and Senior High* The objectives to be tested in each version of the 
Inventory were aelectod from objectives at that corresponding grade level. The 
inventory was designed to indicate general levels of achievement of a student or 
a class and to assess need in the individual or the class in attaining curricular 
goals. 

This report will summarize the development of the Research Edition, the field 
test administration, the process of data analysis » and the development of the 
Fall-72 Version OSAI* Many persons participated in the development of the OSAI, 
contributing their comments and criticisms so that the test items could, through 
revision, be improved. 

In the development of the Research Editioni the test items were written to re- 
present selected objectives and ware rewritten and revised through drafts one, 
t\Wf and three. These activities and the activities related to the field test 
administration will be summarized. 

The test administrators and coordinators contributed input as a result of thair 
experiences during the field test. This data, along with the analysis of student 
scored, was the basis for the revisions of the Research Edition* As a result of 
this input, a fall-72 Version suitable for pre- and pcst-testing was available 
by September, 1972* 

In an at:*:;:.;r to more clearly present the revision process from item conception 
to fall-72 OSAI, the revision process is shown throughout the report by using 
examples of an actual item. The summarization of the total process of revision 
of that item la shown in Appendix B. 

DEVELOPMENT OF RESEARCH EDITION 

Administration Method 

The test items on the inventory could be administered in several ways: a written 
test administered to a group of students, a set of questions administered to 
individual students on a one-to-ono basis, or a combination of these. Considering 
tho probability of the limited academic skills among the EMR students, methods 
other than the traditional paper-pencil test were considered in order to effect- 
ively evaluate a student's progress on any given objective. 

The student could be stimulated to respond to a given question through the use 
of an oral or visual stimulus. This could include any number of materials that 
would be shown to the student or given to him/her so that he/she would be able 
to manipulate them in response to the question or directions. The mode of student 
response could be oral, written, or manipulative* In writing test items, the 
task participants were asked to specify the stimulus and response modes that 
would most efficiently evaluate the student's achievement of a given objective* 

Another factor to be considered in selecting an administration method was tlme« 
An individually administered test would require considerably more time for 
administration than a paper-*pencil test administered to an entire group* 

25 



In naking a decision on the method of test adtalnldtrationi conaldeifatlon would 
have to be given to the effectiveness of the method in relation to the adminis- 
tration time Involved, 

OrRQnization of Content 

In order to better categorize the test Ifc'sms, four areas of development were 
deteraiinad. It was the feeling of the project staff that each of the 73 sub- 
curriculum areas could be classified as one of these types of development: 
academic I personal , social » or vocational* Each of the sub-curriculum areas 
was then assigned to one of the four types of developnent. For Instance^ In 
curriculum area 12, Learning to Understand Oneself, sub-area I, physical 
appearance, would be listed under the peroonal development area while sub-area 
IV, school and peer relationships, would be listed under the social development, 
area* 

Selection of Content 

In order to develop an Instrument that was representative of the 7,000 objectives, 
It was necessary to design a method of sampling that would incorporate objectives 
from all 14 curriculum areas at each grade level* A sampling procedure was 
adopted in which a minimum of one objective x^as selected from each of the 73 
sub-curriculum areas at each of the four grade levels* The sampling procedure 
was not random; objectives were selected on the basis of significance to EMR 
students* A significant objective was one that the task participants felt 
1) vas of academic importance for acquiring basic skills, 2) was of importance 
for personal hygiene, 3) was of social importance in order to function in daily 
living, and 4) was of value in securing and maintaining a vocation* 

Throughout the development process, Mentor's six-digit objective code was 
retained as a key to the historical data of an item* The six-dlglt objective 
code Was a unique Identifier assigned to the objectives os they were developed « 
From the objective code, one could determine the curriculum area and the grade 
level of tho objective. The first two digits of the six-dlglt number represented 
the curriculum area (01-14), the third digit represented grade level (1-4), 
and the last three digits represented the sequence number of the objective* 
For example, objective *^042156" was an objective in the 04 curriculum area 
(area 04 refers to the Reading curriculum) and the second grade level (grade 
level 2 refers to the Intermediate level)* The number 156 is the sequence 
number* By retaining the six-digit number, It was possible to trace an item 
back to its original curriculua area and grade level* 

Draft One 

In following an item from Its conception through to its final form as a field 
test Item, much time was spent in reviewing the item to make It a good criterion- 
referenced item* 

From the obje^^tives that were selected to represent all curriculum and sub- 
curriculum areas, an Item was written that would evaluate the achievement of each 
selected objective* A selected objective was reread and a rough-draft test item 
written* Any special materials to be used as a stimulus condition wer^ to be 
specified in the draft item* Each draft item was assigned an ID number. This 
number had two purposes: to give each item a unique identity and to identify 
developmental areas* For example, Items 101 through 250 would be academic, 251 
through 351 would be personal, etc. 
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An exatipla of an actual Item at the tough**draft stage is tha folloulng}^ 



Item nZQ, Junior High Level 042156 

In those dlctlon^^ty pages, place an *'X^^ by 
each of the words on this li8t« (Get words 
unknown to age level) 

An Information file was set up so that Infotrtiatlon could bo stored on each t^st 
item providing a linkage between the items and the objectives. Tha information 
stored in this file included the following; the three-digit unique test Item ID, 
the curriculum and sub-cutrlculum areas, age level (5 « Primary, 9 Intermediatra 
13 " Junior High, and 17 » Senior High), the developmental area (academic, 
personal, social, vocational), and the modes of stimulus and response* Items 
could be listed sequentially by ID, by curriculum or sub -curriculum areas, by 
age level, or by developmental area (see Figure I). 

After assigning each item an ID number, setting up the information file, refining 
the items to improve wording, and having the items types, they were conslCered to 
be in the final torn of Draft-one (see Figure 2)* 

Draft Two 

The Draft^one items were then reviewed by two consultants who had knowledge of 
and a background with exceptional children, one with the Nisonger Center and 
one as a consultant for stvoial universities in the area of educating exceptional 
children. They reviewed the items for 1) quality of item, clarity, and wording 
the item must be easily understood by the student; 2) appropriateness to age 
level the skill should be within the capabilities of this age group (mental 
and chronological age); and 3) redundancy among instruction areas and age levels 
Their suggested revisions Included assigning an item to a more appropriate level, 
simplifying or clarifying the wording of an item, and converting objectives or 
leairning experiences to test items « Redundant items and items of little value 
were eliminated* They were also responsible for assigning 20 items to each 
developmental area (academic, personal, social, vocational) at each age level, 
a total of 320 items. The items within a group of 20 were sequenced from most 
basic skills to advanced skills. The remaining items were placed In an 
"unasslgncd" category (59 items) to be field tested and possibly used as 
substitutes later* 

After these consultants had reviewed the items, their comments and revisions 
were incorporated into the Draft-one items by Battelle. The Draft-two items 
(320 plus the unasslgned) were set up in a new format that Included the stimulus 
condition, the test question, and the response (s) that would be accepted as 
correct* The stimulus condition could be an oral question or a visual stimulus 
such as a picture, written material » or other special material^^ u^cd to provide 
the condition necessary to administer the question. 

The consultants comment on the example, Item #228, was to clarify the question. 
Incorporating their comrient, the item la the new format was written as the 
following: 



*Item 228 will be used as an example throughout the report to illustrate the 
revision process. For a sutr^ary of the revisions, see Appendix B. 



^»VA special material is an extra device needed to administer the test; this could 
be something the student would look at, handle, or use in some manner. Examples 
-jJ^^ a ball, milk cartons, blocks of wood, crayons, etc. 
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stimulus s dlctlondty» word list 



Question: Using the dlctlondi:y> £lnd and wtlte the 

definition c£ the words on this llst« (Get 
words unknovm to ag© level). 



Response: 



For an overview of the Draft- two process, see Figure 3. 
Draft Three 



Dtaft-two Items were rev;rltten Incorporating the evaluative cotnments of two 
satellite coordinators* They were asked to review the items because of their 
experience teaching and working with EMR's, It was felt they were well qualified 
to evaluate an Item^s value to the EMR as v;eU as determining whether the EMR 
student was capable of accomplishing the objective being testedt 

Along with evaluating the stimulus and response modes, for each item the 
coordinators assigned a method of administration which they felt would be most 
effective in evaluating the objective considering the reading capabilities of 
M?. students. The method of administration could be either individual on a 
one-to-one bauls, or in a group setting. In order for Battelle to determine the 
amount of time needed for administering the group and individual items, the 
coordinators V7ere asked to approximate the maximum number of seconds required for 
a student to respond to each item. 

In looking at the coordinators* comments on the example, Item 228, the following 
evaluative Information was provided: 

1) question unclear - How many words was the student required to locate? 

2) VJhat kind of dictionary would be used? 

3) Stimulus would be visual/oral} Response would be written* 

4) The item could be administered in a group administration* 

Incorporating their suggestions, the item was rewritten as follows: 



Stimulus: Webster's Seventh New Collegiate Dictlonary-- 
p, 354-361 containing words given; worksheet; 
wordllst: gobble, gondola, glacier, goulash* 

Question: Using this dictionary, find the words on this 
list« Write the words and their definitions 
on the worksheet* Gobble, Gondola, Glacier, 
Goulash 



Response: Student will correctly record the words and 
their definitions. Gobble - to swallow or 
eat greedily; noise of the male turkey; Gondola - 
long narrow flat bottomed boat used on canals of 
Venice; Glacier - a large body of ice moving 
slowly down a slope or across land; Goulash a 
beef stew with onion, paprika, and caraway* 

For the field test administration the question was changed slightly to read: 



Using this dictionary, find the words on this list and tell me their 
Q definitions* 



The revised Dreft^two lter.8 were set up into four test verolons according to the 
four age levels (05 > 09, 13 > and 17) • At the suggestloa of the two coordinators, 
the age levels for the four versions were changed to 03, 11, 14, and 17 » These 
ages soctned to represent more realistically the four levels at which EMR students 
are placed s primary. Intermediate, junior high, and senior high* The test 
versions were determined by the four age levals and the Irour developmental areas 
within each age level. Thus, four toot versions resulted, Version 08, Version 11, 
Version 14, and Version 17, with a total of 00 Itetns In each tost version, 20 
items from each of the four developmental areas. Arrangement of the test Items 
within each test version was by method of instruction, group administered items 
together and individually administered items together • 

Additions were then uade to the information file. These Included ^ the method 
of administration, group or individual; the maKlmum number of seconds for response} 
the special testing materials necessary for administration; and the coding for the 
developmental area of an item. Developmental areas were coded as followsj 1 
equals academic, 2 equals personal, 3 equals social, 4 equals vocational* For 
example, any items classified as 108 would be acadfmic items at the 08 level*. 
Items could now be listed by method of administration, by age level within 
developmental area, as well as by curriculum areas. A listing of all special 
materials required for administration could be produced along with the items 
for which the special materials were needed* 

The test items were also reviewed by a professional test developer* He was 

asked to criticize the test Instrument as a whole, assessing the items in terms 

of overall quality. His suggestions Included revisions in wording and format. 

His primary responsibility was the final sequencing of the items for administration. 

The criterion for sequencing items within each of the four test versions were 

1) developmental area, 2) method of administration, and 3) difficulty of items. 

For an overview of these activities see Figure 4. 

Upon completion of Draft- three, production began. Members of Battelle^s staff 
met to design a usable format for the Instrument. It was decided that indi- 
vidually administered items would be printed on individual 8V^ x 11 ' cards which 
could be folded in tha center to allow the teacher to read the question and, at 
the same time, allow the student to view any visual stimuli printed on the card. 
The group administered items would be printed in disposable booklets, one 
booklet per student per test version. There were individual and group admin- 
istered items at the 11 and 14 levels, level 08 was totally individualized, and 
level 17 was completely group administered. The decision to administer the 
items in this manner V7as made after our discussions with the two satellite 
coordinators. The general feeling was that ain's at the 08 level would have 
difficulty handling a test that required Independent reading skills, EMR^s at 
the 11 and 14 levels could handle some items independent of individual super- 
vision, and EMR' 8 at the 17 level possessed reading skills sufficient to complete 
the test with few problems. 

The final step was the actual printing of the individual cards and group 
booklets. The art materials and visual stimuli were completed to our speci- 
fications and the variety of special materials needed for administrtitlon were 
collected. 
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(Field Test section deleted) 



DEVELOPMENT OF FALL- 72 VERSION* 

Before the revision process was begun, several meetings were held between Battelle 
staff and Mentor • Durins those Pr^,$tings derisions were reached concerning the 
degree of involvement of the revision process. The iimediate objective was to 
Supply Mentor with a revised draft of the OSAI in a foraat suitable for Pall 72 
pre-testing and Spring 73 post-testing* Revision process objectives agreed upon 
by Mentor and Battelle were as follows i 

1) for low^relevance items, to verify their relevance to a current, 
revised objective; 

2) to better equalize the number of items per curriculum area by 
establishing a minimum of six items for each curriculum area 
and supplying new items to deficient areas; 

3) to base all item revisions on a clearly defined need for revision; 

4) to specify "correct*' and "partially correct" responses in terms 
of minimum responses and mode of response; 

5) to simplify the individual student's response evaluation sheet; and 

6) to provide a means of summarizing each student's scores by curriculum 
area for the teacher » 

Discussions included suggestions for a new format (individual cards in a looseleaf 
notebook), considerations of administration methods, criteria for item revisions, 
and a format for the item-by-item revision process. 

M ethod of Administration 

A major consideration was the method of administration. A number of negative 
comments had been made in regard to the group-administered tests. Arguments 
against the group-administered test attacked the lack of individual attention and 
the resulting lack of classroom control; the problems encountered by students try- 
ing to work at a group pace; and the frustrations of students trying to success- 
fully complete a test that required reading skills they did not possess* Tho 
major argument for the group-administered test wai: the time factor; coordinators 
felt it Just wasn't possible to test each student individually. They also felt 
most students had sufficient reading skills to complete the test, A comparison 
of data on the Version 17-Group test and the Version 17-Individualized test gave 
no indication that students performed better on a totally individualized instru- 
ment; the levels of difficulty appeared to be comparable on the two Version 17 
tests (see Appendix A, pp. 5-6). After considering the comments of administrators 
and coordinators and the data analyses, the decision was made to change the field 
test administration method slightly. Version 08 would be totally individualized 
as it was in the field administration and Versions 11, 14 and 17 would be partial- 
ly individually administered and partially group administered. The emphasis was 
on simplifying the reading level of all materials, testing reading skills only 



^Involved with the field test and analyses activities was the naming by Mentor 
of the experimental instrument as the **Ohio Special Achievement Inventory** (OSAI). 
Future references will be made to the OSAI instead of the ''test instrument.*' 
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when the objective was the achievement of reading » and attaining an administration 
node that was both effective and easily administered In i\ short period of time. 

leet Mnterlals 

The disposable test booklets would be printed In large typo at all levels and 
administered In a group setting i having the administrator read each Item as the 
test was being taken* Again the test booklets would be color coded to represent 
the four levels* The Individual items would be printed on Individual sheets, one 
Item per page, contained In a loose-leaf notebook for convenience In handling* 
On this sheet would be a list of any special materials needed for administration, 
the test question, and a detailed response evaluation, Indicating correct, partial, 
and Incorrect responses. The art materials and special materials would be in a 
separate package with the art materials (visual stimuli) in some sort of binder 
or notebook. All materials would be, identified with the associated test Item 
number for ease in locating* An agreement was reached that; Bat telle would provide 
20 sets of materials for the individual administration of items and 2800 sets of 
disposable materials* The OSAI would remain in four versions, but the versions 
would be renamed Primary, Intermediate, Junior High, and Senior High. 

Revision Process 

The procedure to be followed for revisions included first grouping all data on a 
particular Item. This was done by assigning each item a folder to accumulate 
historical data on the Item, such as the curriculum and sub-curriculum areas, 
item number, and age level} a copy of the test item; scorabllity rating; co- 
ordinators* and administrators' comments; and statistical data such as the 
response distribution and distribution of response related to mental age. Each 
item was then considered for inclusion on the basis of representation of curricu- 
lum areas, relevance to objective, and quality of Item. Items were grouped by 
curriculum areas, reviewed and revised or sometimes discarded* After existing 
ItemD had been considered for inclusion, the curriculum areas in which deficiencies 
existed could be determined. Tt;o coordinators were asked to review the deficient 
areas and wlte new items so that a minimum of six Items would represent each 
curriculum area. A prime factor in the revision process, then, was being able to 
maintain six items In each curriculum area at each level* 

The greatest need Identified by the test administrators was for more specific 
"acceptable responses The Idea of giving partial credit for responses to Items 
seemed valid because of the complexity of many Items. It would seem of little 
value to evaluate the students* performance on a yes-no basis — yes, he/she has 
achieved the objective perfectly, or no, he/she was unable to give a 100% perfect 
ansv^er. In evaluating the attainment of an objectlvei It would seem reasonable 
to score the student on the degree of attainment so that the amount of Instruction 
still needed to achieve the objective might better be perceived. For these 
reasons, the decision was made to specify "correct** responses, "partial" responses, 
and In some cases, "unacceptable" responses. For scoring purposes, a partially 
correct answer would have half the value of a totally correct answer. Responses 
were Identified as clearly and as specifically as possible* Most responses still 
required teacher Judgment, but an attempt was made to specify the actual activity 
necessary for successful completion of the objective. The minimum requirements 
for both "correct" and •'partial" responses were stated. For the Individually 
administered Items, these responses v;ere Included on the sheet with the question 
and special material requirements (Figure 9). For the Items In test booklets. 
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a separate ^Vesponse evaluation" form waa aet up so that the administrator could 
quickly go through the teat booklets and evaluate the students^ responses 
(Figure 10). 

For a follow-up on tost item 220, the field test provided data In the form of 
cotnments and response distribution. The v.oot emphasized criticism was that the 
Item required reading skills not possessed by Junior High level students. The 
v;ord8 were completely unknown to the students • The suggestion was made that one 
consider the objective of the Item; was the objective for the student to be able 
to read or to be able to use the dictionary? Another criticism was the difficulty 
of evaluating the student *s response; clearer guidelines were needed for adminis- 
tration and scoring. 

The response distribution indicated that 2C% of the students responded "incor- 
rectly," 34% responded '^partially correctly," and 38% responded "correctly." 
Considering the administrators* cocnments, it would seem that too much was required 
of the student for a "correct" response, that the reading skills required were 
greater than those of BMR's at the Junior High level, and that the method of 
administration could be revised for a more valid evaluation. 

After analyzing the field test data of item 228, the Fall- 72 Version of the test 
item remained at the Junior High level but several revisions took place. Item 
228, as it appears on the OSAI, is as follows: 

Ilaterlals ; dictionary. 

Question: Using this dictionary, find the words "carry" and "turkey." 

Read the definitions for each word to me. (Administrators 
may spell words for students) 

Response Evaluation: 

Correct---look up both words in dictionary, locate words, 

and read one definition for each word; 
Partial — locates words, reads at least part of definition. 

The two final revision-process objectives dealt with a format for recording the 
students' test scores. In the field test version, the "Data Recording Form" was 
rather long and complo:c making it difficult to use. This was primarily because 
of the amount of data that was being recorded. A scoring sheet for the pre- and 
post- test need not be that detailed. The principal purpose of the scoring sheet 
was to record the scores of an individual on each item and arrive at a total 
score for the entire test. In fulfilling the objective to provide the teacher 
with students' scores by curriculum area, the test items were arranged by cur- 
riculum area on the scoring sheet as they v;ere on the test. After the completion 
of each set of items in a curriculum area, a sub-score for that set of items could 
be tabulated. A total of the sub-scores would provide the student's total score 
on the test. 

The scoring sheet for the OSAI was abbreviated to a one page format. Included 
on this scoring sheet was information about the student (name, code, class, etc.), 
date of testing, test administrator's code, and the area for recording the 
student's responses. In the Intermediate, Junior High, and Senior High levels, 
the scoring area was divided so that the individual sections and the group sections 
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ITEM 155 



MATERIALS 
Yardstick, foot vulev. 



QUESTlOl^ 

a) What Is this? (yardstick) 
l^hat Is this? (tuler) 

b) How many one>foot rulers does it take to make the length of one yardstick? 



RESPONSE EVALUATI ON 
CORRECT 

Identifies ruler and yardstick as either ruler and yardstick 
or foot and yard; indicates 3 feet are in one yard. 

PARTIAL 

Identified measuring sticks; does not correctly identify number 
of feet in a yard, or cannot identify measuring sticks but can 
identify number of feet in a yard. 

UNACCEPTABLE 



FIGURE 9 
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RESPONSE EVALUATION 



PARTIAL 



UNACCEPTABLE 
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Indicates two concepts relative to 
ecology: Provides oxygen, adds 
beauty, provides food for man and 
animals, useable resource, pre- 
vents soil erosion* 

EKprcsses In one word their 
beauty---relatlve to color, 
shape or grace* 

Ihrltos four lines that represent 
an original poem (with one major 
thought) with the last word of the 
second and fourth line in rhyme • 



Indicates any two acceptable forms 
of the fine art;i*--use liberal 
Judgment; e.g., painting, drawing, 
music, dancing, photography, 
ceramics, macram«, etc. 

Indicates any 3 different kinds, 
styles, or forms of music, e.g.: 
rock & roll, classical, ballad» 
opera, Jozz, pop ov: popular, folk, 
march, country v;estern, bluegraes, 
symphony, religious. 

Indicates concept of recycling 
or blo^degradlng. 

Indicates tv;o things relative to 
food» clothing, and shelter. 



Indicates two ways such as, 
classified (want) ads, employment 
agencies, word of mouth. Job 
counselors, bulletin boards, radio 
(news media), parents, friends > 
school. 

Bartender 21 

Deliveryman---18 

Circles letter 

Circles letters ''A" and 't)''. 



Indicates only one 
correct concept. 



None« 



IJrites four seem* 
Ingly unconnected 
lines but rhymes 
the last v;ord of 
the second and 
fourth lines. 

Indicates only 
one acceptable 
art form. 



Indicates 2 
correct types 
of music. 



Hone 



Indicates only 
one correctly. 



Indicates one 
only correctly. 



Responses such as 
"smell good," 
"sound pretty." 

Urltes only two 
words that 
rhyme. 



Names o^ instru* 
ments, songs, 
musicians, or 
dance steps. 



Particular and 
non-'basic Items 
such as TV, re- 
cord player, etc, 
or simply 'Wney" 



Answers only one 
correctly. 

None. 

Circles one letter 
only--clrcles "A" or 
"D", not "A" and "D" 



FIGURE 10 
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could conveniently be scored separately « (Figure 11) • Since the lndc<vldual 
scoring sheets w^re to be returned to Battelle for analysia and Incluelon In 
the data badci a separate ''OSAI Clase Profile*' sheet vas designed to give the 
claasroora teachers insmediate feedback on their students* This profile sheet 
acted as a channel of coimnunication betv;een the administrator and classroom 
teacher • The administrator was to transfer the sub-scores (scores from each 
curriculum area) of each student onto the profile sheet. The profile sheet was 
broken dovm into the fourteen curriculum areas by individual students with total 
scores also given* Information was given on the maximum scores possible for each 
curriculum area of each test version and on the total scores possible for each 
of the four test versions (Figure 12). this provided the classroc^m teacher with 
the Information necessary to (I) evaluate each student's performance in each cur- 
riculum area, thereby identifying areas in which the student was weak, (2) compare 
students within the classroom with each other, and (3) score the students' pro- 
gress by comparing individual scores to the maximum possible. 

Fall-72 Version OSAI 

Production was completed when test booklets were printed} individual sheets of 
items assembled in looseleaf notebooks; art materials (visual stimuli) completed 
and arranged by item; and special materials acquired, labeled, and boxed. The 
scoring sheets and disposable worksheets to be used by students were bound in 
pads of 25 for the convenience of the test administrators. Twenty sets of 
materials for test administration were provided to Hentor. Each set included 
the following s 

*• One notebook of individually administered Items, one per page, with 
special materials and response evaluation specified. The Items were 
color coded and grouped by test version. Also included were the 
response evaluation sheets for group items. 

- One set of art materials (visual stimuli) per test version. 

- One set of special materials which contained all other equipment 
not Included in the art materials. 

A total of 2850 disposable test booklets at the Intermediate, Junior High, and 
Senior High levels were provided to Mentor. Other materials prepared for 
Hentor included the disposable worksheets, the OSAI Scoring Sheets (900 per 
level), and the OSAI Class Profile sheets. 

The testing materials were distributed to test administrators at a brief one-day 
training session held at Battelle In September 1972. Seventeen research 
assistants had been employed by Mentor for the pre- and post-testing; the pre- 
testing was to be done in the Fall of 1972 and the post-testing to be done In 
the Spring of 1973. Coordinators were invited to participate in the session 
so that they would become more familiar with the materials and administration 
techniques of the OSAI. The agenda included a brief background of test develop- 
ment and the revision process, scanning the test materials (booklets, individual 
items, special materials), and a brief review of the test Items which Incorporated 
questions and administration techniques. 
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APPEMDIJC A : 
FIELD TEST DATA ANALYSES 
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COl^CLUSIOtl 



Tho OSAI is currently being administered in the eight satellites as well as 
the Dayton and Columbus areas. It Is anticipated that another revision process 
will take pace in the Spring of 1973, At that time the data received in the 
form of students' scores, administrators' evaluative comments, and feedback 
from the coordinators v/ill be analyzed to determine the extent of the revision 
process. 

Very little qualitative data has been received to date. A few administrators 
have sent in coiments in reference to particular test items. Au administration- 
cost study was done by two administrators! The results were a comparison of 
total costs of administering the four versions of the test and a breakdown 
shov;ing time ejcpenditures for various activities, such as scoring, travel, and 
testing time. 

The comments received so far suggest that minor revisions may be required for 
selected items in the following areas; 

• improvlnc. clarity 

• making items more. specif ie 

- evaluating the value of ''partial'' and "correct" responses. 

Other areas of revision may be identified as additional Information is re- 
ceived from the test administrators and coordinators* 

FIELD TEST DATA ANALYSES 

This report summarizes work done in the analysis of data collected during the 
first field test of the four levels of the OSAI: 08, II, 14, and 17. Pre- 
ceding this analysis summary is a brief discussion of issues connected with 
the question of v/hether to emphasize criterion or norm referencing in the 
ongoing revisions of the OSAI. 

The OSAI - Criterion and/or Worm Referenced? 

Since Ebel* and Glaser^^ have made the subject visible to most tests and 
measurement specialists, there has been increased attention given to criterion 
referenced tests (CRT^s), which relate test performance to absolute standards 
rather than to performance of others. 

The arguments for and against CRT*s are many. For example, their emphasis on 
individual proficiency would force instructional staff merabers to focus on both 
the process and outcomes rather than process alone. Instructional means would 



* R. L. Ebel, "Content Standard Test Scores," Educational and Psychological 
Measurement, Spring, 1962, 15-25 

**Robert Glaser, "Inslinctlonal Technology and the Ho^surement of Learning 
Outcomes: Some Questions/' American Psychologist, August, 1963, 519-521* 
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bo Judged by the cnd6 achieved* The teacher , student » and parent vrould know 
better vhat the student has learned and ohat can or should be done nexti 
CRT^a would appear also to improve student attitudes through Increased 
emphasis on the student *s mottlvatlon to better hlraself and not his classmates • 

On the other hand) certain disadvantages are often cited « Verfotmance 
objectives must be stated in behavioral toriDs - thus, performance in the 
affective domain would be hard to assoesi and objectlvao involving the re"* 
tention and transfer of learned skills might be sUghted« The two key areas 
of difficultyi however, appear to be specification of the universe of tasks 
to be learned and determination of proficiency standards • 

Both of these two last mentioned ptobXeti\ areas are quite relevant to the 
construction of the OSAIi Concerning the first, one must identify a small, 
manageable number of instructional objectives to be taught and provide 
several (say, 4-8) test items that will provide a reliable and valid determina- 
tion of their attainment by the student* One cannot simply define a universe 
of possible test Items and randomly sample from it*"^ Concerning the second 
difficulty, the choice of a proficiency standard is quite arbitrary* IJhether 
a student's performance is good enough to allow him to continue instruction 
in nev; skills is, in the final analysis, a matter of Judgment. It should not 
be overlooked, however, that these two difficulties hold also for norm refer- 
enced tests (NRT*s)» NRT's must have content validity and the determination 
of a proficiency standard remains even though it is couched in normative 
terms, e«g,, grade equivalents and percentiles* 

In summary, it v7ould appear that the point is not whether to construct a CRT 
or NRT, but to construct a test which contains the advantages of both; an 
honest effort to allow the comparison of a student's performance to a standard 
which is meaningful to the student, teacher, and parent* This challenges one 
to construct a test vjhose score, or subscores, is reliable and valid In diagnos- 
ing the individual student's strengths and weaknesses and in comparing groups 
of students* 

The major problem facing the effort to construct the OSAI is to carry out 
analyses of items in order to discover what items are good in both a CRT and 
NRT sense* In constructing NRT's, one typically finds three quantitative 
indices for each item being examined: (1) the item*s difficulty (percent 
passed), (2) the item's correlation v;lth some total or subtotal score, and 
(3) the itero^s loading on one or more "factors" identified by the mysteriously 
misused technique knotm as factor analysis. One also finds total or subtotal 
scores being assessed for both reliability and validity, quantitative Indices 
always being presented for reliability and often for validity (the validity of 
an achievement test is usually a matter of eK^>ert Judgment - this is not to 
preclude, however, studying the relationships among achievement measures of the 
same subject). 



^Jane Loevlnger, "Person and Population as Psychometric Concepts," 
Psychological Review, llarch 1965, 143-155 • 
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A mdjot issue has been made by sotoe to find or invent parallel , quaotitative Indices 
of an Item's ^Vorth'* for CRT teots** It Is Interesting that several of these 
ettetnpto have led to very high relationships between an item's worth a la CRT 
methods and the traditional NRT methods. This is not surprising in view of the 
fact that extremely difficult or easy items on a CRT test administered in a 
way and at n time unrelated to the use of the underlying instructional objectives 
should be considered suspect (at least in terms of the level of difficulty of the 
whole test). Similarly, the content validity of CRT items should be quite high 
and, if grouped into content meaningful subtests, they should correlate well 
with each other and their sum « thus assuring high internal consistency reliability. 
Also, any factor analysis done with due care to the use of the appropriate analytic 
model should yield high loadings on common factors for items testing the aamo 
objective or level of objective. 

Employing the above reasoning, the preliminary analysis of the OSAI items for each 
of four levels (08, 11, 14, and 17) was carried out in the following whinners 

a All data processed for each item was checked (looking for errors in 
scoringi keypunching, and the extent of missing data)* 

a Response distributions for each item were examined (the scoring 
system used for this first field administration was l>*incorrect, 
2«partially correct, 3«correct)» 

a Factor analyses, using several common models, where contemplated, 
but were not carried out due to the small number of examinees relative 
to the number of items and the relatively low benefits anticipated 
for high costs* 

e Item**total correlations were calculated between each item and the 
total score in (1) the curriculum area to which the Item belonged, 
and (2) the core area to which it belonged. 

Several other analyses were performed to gain additional Insight into the 
validity of the items and two curriculum areas; reading and arithmetic* 

a Response distribution by mental age (in years) were examined for each 
item* 

a lixe distributions of mental age and chronological age for the examinees 
taking each level were compared. 

0 The reading and arithmetic curriculum area subscores on the OSAI were 
correlated with the Reading and Mathematics scores of the Metropolitan 
Achievement Tests (llAT)* 



*Sed, for example, Hsu, TsoH^hi, •'Empirical Data on Criterion-Reterenccd Tests," 
paper presented at the Annual Meeting of the American Educational Research 
Association, New York, February, 1971; W* J* Popham and T* R* Husek, "Impli- 
cations of Criterion-Referenced Measurement,'* Journal of Educational Measurement , 
Spring, 1969, 1^9; and R« C* Cox and J* S* Vargas, "A Comparison of Item Selection 
Techniques for Norm-Referenced and Criterion-Referenced Tests," Report No* BR-5- 
0253. Learning Research and Development Center, Pittsburgh University, i^^bruary, 
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Resultfl 0^ It^, and Vflltdation Analyaeo 



The chockd of alt processed data were done and few errors vore discovered and 
corrected • Almost all errors were keypunching errors ** a tribute to the care 
and capability o£ tha field test administrators, 

The response distributions for each lovel wore examined » (A copy of the computer 
printout displaying these distributions for each level is contained In Reference A*) 
Concerning Level 08| the items appear to have been relatively easy overalls 
Thlrty-^two of the 86 items (or 37%) were passed by at least 80 percent of the 
examinees. Only four (or 5%) were passed by less than 20 percent of the examinees. 
Of additional interest > especially with respect to item revisioni were the number 
of items scored partially correct. Items scored this way were sufficiently multi- 
faceted or ambiguous that the test administrators felt that the examinee did not 
fall the Item but nor did he pass it* There were 20 (or 23%) such items for which 
at least 20 percent of the examinees were scored **partifilly correct.^' In summary, 
60 percent of the items on Lovel 00 were at leaot partially correct for all 
examinees* 

On Level lA'^v, 10 (or 12%) of the 96 items were passed by at least 80 percent of the 
examinees. Sixteen (or 17%) were passed by less than 20 percent. And 45 (or 47%) 
were scored partially correct. Therefore, 59% of the itmns o\\ Level 14 were at 
least partially correct for all examinees. 

On Level 17*^, only tyo (or 2%) of the 94 Items on Lovel 17 were passed by at least 
80 percent of the examinees. Twenty (or 21%) were passed by less than 20 percent. 
And 45 (or 48%) were scored partially correct. Thus 50 percent of the items on 
Level 17 were at least partially correct for all examinees. 

In summary, ignoring partially correct responses, the difficulty of the OSAI 
appears to increase with age level the 08 level being the easiest and the 17 
level the hardest. Including partially correct responses, Level 11 appears to 
be the easiest. UMfortunately, the nature of the partially correct response is 
ambiguous. 

Probably the most defensible explanation of this increase In difficulty with age 
level is that the gap between mean mental and chronological ages of the examinees 
tested Increases as the level of the OSAI goes up. The difference between the 
two means for Level 08 examinees was CA-MA «9*0 -6.2tt2.8. For Level 11 
examinees it was 12,2 - 8.7 » 3.5. For Level 14 it was 14.5 - 10.3 « 4.2; and for 
Lovel 17 it was 17.0 - 11.7 « 5.3. That there is indeed a relationship between 
mental age and item response is supported by the cross tabulations of mental age 
and tjrpe of responses presented for all items for each level of the OSAI in 
Reference B. For the great majority of the items, the item becomes easier with 
Increasing mental age« 

The hypothesis that switching from individual to group administration as the 

OSAI's age level increases explains the increasing difficulty does not appear 

to be a strong candidate in that the revised version of Le^el 17 was Individually 



*The revised versions of Levels 14 and 17 are not discussed here. There appeared 
to be no systematic difference in their response distributions from the original 
versions. 
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administeted and the level of difficulty did not appear to go dotm overall » AlsOi 
the Gussestion thac the higher levels require trore reading skills Is not supported 
well for two reasonfl. Firsts tho group administered itews were read out loud as 
well as being presented visually in booklets* Second, the correlations between 
the CSAI total score and the ILW Reading score (see Vable 1) is not consistently 
high at either the 03 or 17 levels • At all Iwels, the OSAl total score corre- 
lates with the ^tAT reading score noticeably v;ore than It does with the ItAT 
Mathematics scote in only one case - for those sttidents \;ho took Level 2 of 
the MAT, In all other cases except one, its correlation with llAT Reading was 
iQBs than with IIAT mathewatics. Therefore, the OSAI cannot be said, on the basis 
of this evidence, to be predominantly a reading tost «• even at the higher age 
levels. 

One should also note in Table 1 that the OSAI Arithi/.etic curriculura area subscore 
always correlates substantially with the IIAT Hathematlcs score. The correlations 
between these two instrunents in reading, however, are substantially high (greater 
than 0.400) only for the 11 and 14 Levels of the OSAI. The two lo--^ correlations 
between the two reading scores at the OSAI Level 17 can be partially explained 
by the fact that the OSAI Reading score is based on only one item. That it 
would correlate in the 0.300*6 with the MAT Reading score is remarkable 1 The 
low reading correlation at the OSAI Level 00 may be due more to the total lack 
of relationship between the Level 1 IIAT Reading score and the total OSAI - all 
three of these correlations are virtually ssero. 

Finally, for a look at all item^total correlations within the OSAI, for each 
core area (Academic, ?ersonal, Social, and Vocational) and each curriculum area 
(numbered C01-C14 in correspondence with the original numbering of curriculum 
areas), the reader Is referred to Ueference C. In most cases, these correlations 
are substantially positive. The exceptions will be noted in the revision of the 
four levels of the OSAI. 
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n¥^se p/^^ss €04^4t^ 

MISSING FROM THE DOCUMENT THAT WAS 
SUBMITTED TCTeRIC DOCUMENT REPRODUCTION SERVICE. 



Materials Released to Tom Nof f singer i October 3i 1972 
References Description 

A, .Computer printout of response distributions for each 
item on each level of the OSAI (Prepared August, 1972) 

B, Computer printout of response distributions by mental 
age for each item on each level of the OSAI (Prepared 
August, 1972) 

C, Computer printout of the item-total correlations for each 
core and curriculum area on each level of the OSAI 
(Prepared August, 1972) 



46 



AP?EWDIX B: 
REVISIONS OF TEST ITEII 220 
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RSVISIOUS OP imi 220 



An exatftple of an actual Item at the youRh-draft stag© is the following s 
Itetn i?22Q, Junior lUgh Level — -042156 

In these dictionary pages, place an X by each of the words on 
this listt (Got words unknown to age level) 

ghe Consultants^ Revisions 

The consultants' comment on the example, item ^^228, was to clarify the question. 
Incorporating their comment, the item in the new fortiiat was written as the 
following: 

Stimulus: dictionary, word list 

Question: Using the dictionary, find and write the definition of 

tho words on this list* (Get words unknown to age level •) 

Response; 
The Coordinators' Revisions 

In looking at the coordinators' comments on the example, item 220, the following 
evaluative information was provided; 

(1) question unclear - Hov; many words was the student required to locate? 

(2) l/hat kind of dictionary would be used? 

(3) Stimulus would be visual/oral; Response would be written* 

(4) The Item could be administered in a group administration* 

lacorporating their suggestions, the item was rewritten as follows; 

Stimulus; Uebster's Seventh Hew Collegiate Dtctlonary---pp* 354-361 
containins words given; work.^heetj wordllst: gobble, 
gondola, glacier, gouliish* 

Question; Using this dictionary, find the words on this list. Write 
the words and their definitions on the worksheet* Gobble, 
Gondola, Cl^cler, Goulash 

Response; Student x^lll correctly record the words and their definitions* 
Gobble---to swallow or isat greedily; noise of the male turkey; 
Gondola — long narrow flat bottomed boat used on canals of 
Venice; Glacler---a large body of ice moving slowly down a 
slope or across land; goulrtsh---a beef stew v;lth onion, 
paprika, and caraway* 

For the field test administration the question was changed slightly to read; 

Using this dictionary, find the words on this list and tell me 
their definitions* 
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ReflUlts of the Field T^iQfc 



For a follow-up on tedt item 220, the field tost provided data in the fon^i of 
coiments and responae distribution* The most er^iphaoized orltloicm waa that the 
iter.t required reading akiUa not poaaessed by Junior High level students* The 
words were completely unknotm to the students* The suggestion was made that one 
consider the objective of the itemj was the objective for the student to be able 
to read or to be able to use the dictionary? Another criticiam was the difficulty 
of evaluating the student's responoei clearer guidelines were needed for admin- 
istration and scorings 

The response distribution indicated that 28% of the students responded " incorrect ly;* 
34% responded "partially correctly/^ and 30% responded "correctly*" Considering 
the admlnlstratoro^ consents, it would seem that too much was required of the 
student for a "correct" response, that the reading skiUs required were greater 
than those of EMR"s at the Junior High level, and that the method of adminis- 
tration could be revised for a more valid evaluation. 

OSAI Version Item 228 

After analyzing the field test data of item 228, the fall-72 Version of the test 
Iteta retnained at the Junior High Level but several revisions took place* Item 
228, as it appears on the OSAI, is as follox7s: 

Haterials ; dictionary 

Question: Using this dictionary, find the words "carry" and "turkey*" 
Read the definitions for each word to tne« (Administrators 
may spell words for students*) 

Response Evaluation! 

Correct look up both Xirords in dictionary, locate words, 
and read one definition for c^ach word; Partial locates 
words, reads at least part of definition^ 
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ObJeotlvo I IB Flold test the evaluation design by Auf^u8t> 1972 * 

The record of the completion of thle objective can bo found in the follovlng 
report or the Data Baae System for EUR Program Development. 
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SECOND-YEAR REPORT 
on a 

DATA BASE SYSTEM 
for 

EMR PROGRAM DRVELOPMENT 
from 

BATTELLB CENTER FOR IMPROVED EDUCATION 
January 31, 1973 



The Meut^r Public Schools awarded Battellc's Columbus Laboratories a contract to 
provide technical advieory services In the area of computer data processlngi and 
research and development setvices toward the design of an experimental achievement 
Inventory to assess the relative attainment of behavioral objectives by educable 
taontally retarded (EMR) students In the different program models of their Title III 
project, *'EHR Program Models Development" » 

The project effort was oriented toward two primary tasks i 

1) The first atage of the design of an achievement inventory, consisting 
of ttms that would be representative of the behavioral objectives 
generated during the first year of the project, and 

2) The design and implementation of a computerized data base for the evalu^* 
at Ion of EMR programs. 

Under the first task, Battel Je-Coluoabus was responsible for the following work* 

1) Planning the content of each level of the achlevemont Inventory, 

2) Writing test Items for each level, and 

3) Designing the first draft of each level of the Inventory* 

Under this second task, Battellc-Columbus was responsible for two areas of works 

1) The design of a computerised data base, and 

2) The development of computer programs required for using the data base. 

This report presents a summary of Battelle-Columbus* effort conducted under the 
second task. A separate report describes the effort directed toward the develop* 
ment of the EMR achievement Inventory. 

The objectives of the data base system are to provide an efficient means of 
storing data during the project and to facilitate the retrieval of data for a 
variety of applications. In describing how the current data base system meets 
these objectives, this report will first focus on the data base organization, 
then on data storage and retrieval, and finally on each of the principal data 
files in the system. Also Included with this report Is an Appendix containing 
examples of the data'Collectlon forms and Instructions on forms control # 



Data Raae Orf^anlzatlon 

For the evaluation of EMR Program Models, the analyslr will deal with approximately 
2^00 atudents In 200 classrooms distributed over the State. 
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In addition, the analyola wlU inoludd data collected continuously over a throe- 
year period* A6 a result » the data base organlssation must facilitate the inte- 
gration of date from ^ large variety of sources over a long period of tltte# The 
basic objects of analyfiis in the evaluation ure the students » teachers » and 
classrooas* Similarly» the sources for the data to be analyzed are the students> 
teachers, and claosroonis* It is the objective of the Acta bese organization to 
link the sources and objects of analysis in a systematic fashion* If one were 
only dealing with an Isolated classroom and collecting data on student per- 
formance in that classroora and analyzing it, then the design of a data base 
would be superfluous* ^'bwever, when there are almost 200 classrooms which are 
operating independently, but must be evaluate^, on a common basis, the need for 
a data base design is imperative. It la the purpose of this document to describe 
the data base design for this evaluation. 

Figure 1 shows the eight basic types of files in the data base design. These files 
include Class descriptions, financial data, class enrollment lists, teacher de- 
scriptlono, student performance data collected at different points in time, 
student descriptions, OSAI item performance data, and data from the behavioral ob- 
jectives field test. Each of these files is described in more detail in Table 1 
and later in this report* The remainder of this section and the next will deal 
with the interrelationships of the files and the processes by which data is stored 
onto and retrieved from the files* 

Several characteristics affect how the files In the data base are interrelatedt 
These Include 1) the volume of data to be stored, 2) the frequency of adding new 
data ("updating"), 3) the amount of data processed at one time, 4) the timeliness 
required in reporting any changes in the data base, and 5) the level of detail 
of tho data to be stored* Based on the experience gained in the first two years 
of this project, the following values apply: 

1) volume is estimated to reach a maximum of 4*32 million characters, 

2) the frequency of updating is twice a month, 

3) the amount of data processed in a single update is less than 500 records, 

4) the frequency of reporting Is once a month, and 

5) the level of detail of the data is very specific, little summary data 
is stored* 

These general characteristics are further summarized for each file in Table 1* 
Data Storage and Retrieval 

The placement of data Into the data base involves six data^-collection forms* Bach 
of three forms correspond to a pavfclcalar data file while the remaining three 
forms affect i^cveval files simultaneously* 
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The three specific formd Ate the financial data«coUectlon formi the OSAI 
scoring sheet, and the objective field test fom ("Form 2")* These forms update 
the Financial Data Fllej the OSAI Item Performanpo Filei and the Behavioral 
Objectives Pile, respectlvoly. 

The retraining throe data forms are the class and teacher description form ("Form 4"), 
the student deacrlptlon form ("Form lA"), and the student performance form 
("Form IB^Oi Those forms add new classes, add new teachers, update enrollment 
lists, update student descriptions » and add new student performance data* The 
class and teacher description fora, for example, adds a new teacher to the teacher 
description file, and assign a series of student codes for the class in the 
student description file. Examples of the forms and the instructions for their 
use are given In the Appendix* 

Figure 2 shows the placement of data into the data base in terms of the files and 
data collection forms* A unique feature of the ayst^m is the computer printing 
of forms lA and IB, the student description and performance data forms. In 
this process, Form 4, Che class and teacher description form, Inltiateo the 
assignment of atudent codes for a particular class* Once the codes are assigned » 
a blank Form lA (Student Description) is printed by the computer for the new 
class. The names of the students in the new class are filled in by the teacher 
at the class site and the completed Form lA is entered into the data base, This 
entry of Form lA ccmpletes the student descriptions for the new class and a Form IB 
(Student Performance) is printed by the computer* This Form IB is returned to the 
class site for entry of performance data. At the same time, a review copy of the 
completed Form lA Is printed by the computer for the teacher's records. This 
review copy also permits validation of the data stored in the files, lleanwhile, 
when the Form IB (Student Perfomance) is completed and returned to Battelle, the 
data is entered into the data base and a review copy is prepared. The review 
copies of Forms lA and IB not only permit validation of the data but also can be 
used to add make-'Up data and indicate changes in classroom enrollment. The process 
of filling-in Form lA and IB, reviewing, and changing as required continues in an 
iterative fashion throughout the project. In the meantime, data is also being 
collected on the costs for each class, on OSAI item performance, and on Behavioral 
Objectives performance. 

The retrieval of data from the data base will take many forms. One form of 
retrieval is the selection of data given the identification of the class, teacher, 
or student. This form requires only that one specify the desired class, teacher, 
or student identification code. The principal reason that one may do this 
directly is to verify that a particular class, teacher, or student was in the 
data base. Once the particular item is retrieved, the data for that particular 
class, teacher, or student can be Inspected and changed if desired, A second 
form of retrieval is the selection of data given the characteristics of the data. 
In this form, the values of data elements detetmine which classes, teachers, 
and students are retrieved. For example, data on student performance may be 
retrieved for those students in model types 1 and 5 for rtiral and urban geo- 
graphical settings. This 4-way comparison can then be made on student per- 
formance data collected in model types 1 and 5 for rural and urban geographical 
settings. 

Figure 3 shows the various ways in which data retrieval can occur. To the 
evaluator, retrieval can be based on the following parameters: 
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!• Identification Codes 

a* Claoa 

b« Teacher 

c. Student 
2. Data Charnctoristics 

a« Model 

0. Costs 

d* Teacher 

0* Student Performance 

£« Student Doacrlption 

As shown In Figure 3, vhen retrieval Is based on characteristics of the data, the 
files themselves provide the identification codes which link classes, teachers, 
and students. For example, if one wishes to retrieve student and teacher data 
for model types 1 and 5, the following process would occur t 1) the Class 
Description File would be searched for model types 1 and 5, 2) the class idonti- 
flcation codes for model types 1 and 5 would be to retrieve student identification 
codes from the Class Enrollment File, 3) the student Identification codes would 
then be used to retrieve student performance data for the time period of interestj 
and 4)' teacher identification codes from the Class Description Pile would be 
used to retrieve data from the Teacher Description File* An alternative process 
would require the searching of the Student Performance File for students In 
model types 1 and 5, Included in these student records would be the teacher 
identification codes. The decision to use one of the two alternative processes 
described above or any other alternative for an evaluation retrieval would 
depend on the size of the various files, the completeness of the data, and 
similar factors. 

Class Description File 

The purpose of this file is to record non-varying Information about each class 
in the program evaluation. This is stable information which applies to the 
class indfipendently of class enrollment, Infonnatlon from this file can be 
used to report on the status of the project In terms of model types, locations, 
and teacher. 

This file also contains the historical record of all teachers who were associated 
with each class. A Hot is maintained for each class indicating the Code, 
Eutry Date, and Exit Date for each teacher who had taught that class during 
the program evaluation. This record provides the linkage for associating each 
class of students with the appropriate teacher at a given point in time. 

The Class Description File is organized by class and each record contains the 
following principal elements: 

1. Class Code (Access Key) 

2. Date of Entry Into Project 

3. Date of Exit from Project 

4. Date of Last Recording of Infomatlon on this Class 

5. Location Codes 



b. 



c. 
d. 



Satellite 
District 
School Building 
County 



ERIC 



58 



6» Model Infortnatlon 

A* Model Neme 

b. Model Cede 
7« Geographic Setting 

8, Cumulative List of Teacher Codes with Entry and E^lt Dates 

The coded used for county > school district > and school building are Identical 
with tho Identification codes used by the State Department of Education* This 
Insures compatablllty with the existing State Information system for the 
possible purpose of utilising other dletrlct and school data which is collected 
by the State* All other codes are unique to this daha base* 

F inancial Data File 

The purpose of this file Is to accumulate cost Information by school for evalu- 
ating program raodela* This file Is currently ur\der development* 

The Financial Data File will be organised by model/class and will contain the 
following principal elements j 

1* Identification 

a* Class Code 

b* Date Prepared 

c* Geographic Setting 

d« School Building Code 

e« Diotrlct Code 
2* Gcneial Information 

a* ADH of all students in district 

b* ADM of EMR students in district 

c* ADM of all students In EMR classroom 

d. ADM of EMR students in EI^ classroom 

e« Total classrooms in district 

f • EMR classrooms In district 

g. Total classrooms in building 

h* EMR classrooms In building 
3* Financial Summary 

a* Administration 

b. Staff Support 

c* Supervisor of Instruction 

d* Direct instruction 

e* Auxiliary Services 

f* Operations and Maintenance 

g% ;?otal Model Expenditures 

h* ADM of EMR Students 

i* Average cost per EMR student 

Class Enrollnent File 

The purpose of this file Is to maintain a dynamic record of current and past 
class enrollment* This file records the codes of all students who were enrolled 
in each class* Each student code is identified with the dates of entry into 
and exit from the class* For each class the list of student codes is always 
cumulative with current students Identified as having no exit date* 



Using the dated, cumulative list of students for each class > class enrollments 
can be recreated for any point in the history of data collection* 



thd Class Enrollmont File Is organized by class and contains the following 
principal elements: 

U Class Code (Access Key) 

2» Cumulative Nunibei: o£ Sto^ants in Class 

3* Cumulative List o£ Student Codes with Entry and Exit Dates 

Teachey Descript ion gile 

The purpose of this file la to record Information describing the teacher which 
will be of value in performing the proeram evaluation. This flic contains 
information which not only identifies tie teacher, such as name, but also contains 
information pertinent to the analysis of prograius. Examples Include teacher 
experience and education* 

The Teacher Description File is organized by teacher and contains the following 
principal elements t 



1* Teacher Code (Access Key) 

2* Current Class Code 

3* Current Model Coda 

4t Date of Entry into Project 

5. Date of Exit from Project 

6« Date of Last Recording of Infor«Tiation on this Teacher 
7» Name 
8« Sex 

9» Date of Birth 
10 • Experience 

a« Present Uodel 

b^ Total EMR Teaching Experience 

c» Other Teaching Experience 
11 • Education 

a* Degree Level 

b» Area of Major 

c. Area of Minor 



Student Perfomanoe File 



The purpose of this file is to record the performance measures which are collected 
for each student* Many different copies of this File will exist, one copy for 
each data-collection period* It is this file which will be used as the principal 
source o£ information for describing the effects of different models on student 
performance. 

The information recorded in this file is expected to vary over several data 
collection periods* For each period, a new copy of the Student Performance File 
will be created, recording only the performance data collected during that period. 
The first period is fall-winter of l?72 and the second period is spring-summer 
of 1973. Additional recording periods will be based on the availability of new 
data and the amount of time required to record performance data on a large number 
of students* Ideally » one would like to specify a narrow time span in which 
identical measures v;ere collected on all students* However, because of the large 
numbers of students in widely scattered classrooms » this ideal Is not possible* 
Instead, general periods of data collection are used and performance data is 
accumulated by period* On the average, a large proportion of ths students will 
have been measured for each and every data collection period » 
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Thus, there is a definite functional relationship between the number of periods , 
the length of each period i the total number of students tc^sted for a given period » 
and the number of students who vere tested in each and e^^ory period* I^or this 
data base, the decisions regarding the collection porlcdo nan be made Independently 
of any data base requlrccavants. In other words , there are no constraints imposed 
by the data base design on the number of periods » the time between periods, or 
how many students would be included In any one period # 

Once several Student Performance Files have been created for different performance 
periods, the data can be analyzed in several ways. Each file can be analyzed 
Independently, describing student performance by program model, for example. Tt70 
or more files, each representing a different period ^ can be aggregated separately 
and compared* Finally, tv7o or more files can be mersed to sort out wlthln-student, 
between-period factors* 

The Student Performance File Is organized by student and each record contains the 
following principal elements? 

1. Identification by student code (access key) 

2. IQ Test 

a. Performance Sub -Test Score 

b. Verbal Sub-Teat Score 

c. Full-Scale Score 

d. To3t: Identification Cede 

e. Date of Tost 

3. Achievement Tost Data 

&4 Reading Standard Score 

b. Mathematics ^ Standard Score 

c. Level 

d. Date of Test 

4. OSAI Data 
a* Level 

b« Raw Scores for 14 Curriculum Areas 
c. Date of Test 

5. Collection Period Information 
a* Class Code 

b« Modal Code 

c« Teacher Code(s) 

Student Description Fil e 

The purpose of this file is to record characteristics of each student which 
are not expected to change from one performance period to the next* This is 
fairly stable Information which can be used to describe each student in the 
system, to set various independent variables for analysis, and to assist in 
locating students if only limited Information is available about them. 

Since there any many performance periods > it would be waoteful to reproduce 
the non*varying information from period-Co-perlod. It is the purpose of the 
Student Description File to record the non-varying information in one place. 
By eliminating redundancy, the process of changing portions of an existing 
student description is greatly simplified. It is sometimes the case, for example, 
that family names are changed and making this change In a centralized student 
description record is much simpler than changing all the records associated with 
that student. 
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Slnca much of the Information in the Student Description File is descriptive of 
the characteristics of the student, this same Inforraation can be used to stratify 
students for performance analysis • For example, a sweep through the Studont 
Desorlptten File can separate students based on thelv entry data into the EMR 
Program. The codes for these students could then be used to select the appropriate 
performance records for analysis. Even the nunbers of students selected prior 
to analyf?is has Inforriatlve value to the analyst. This kind of information could 
be especially itpportant in trying to achieve balanced groups when selecting 
students for testing, for example. 

The Student Description File can also be used to prepare alphabetized lists of 
all students in the program* This greatly slmpllfleo searching for students whose 
code Is missing or name Is Incomplete. This file is Sta<;e-wlde and Independent 
of the student's satellite location. 

The Student Description File is organized by student and each record contains the 
following principal elements? 

1. Student Code (Access Key) 

2. Clasa Code 

3. Model Identification Code 

4. Date of Student Entry Into the Program 

5. Date of Student Exit from tho Program 
6« Student Name 

7 . Nickname 

8. Sex 

9. Race 

10. Date of Birth 

11. Date of Student Entry into Special Education Classes 
OSAI Item Performance File 

The purpose of this file is to accumulate student performance on the OSAI by test 
Iteta. The resultant data will be used to compute curriculum-area scores for 
Insertion Into the Student Performance file and to perform Item analysis of the 
OSAIt 

The OSAI Item Porfonaauce File Is organized by student and contains the following 
principal elements; 



1. Student Code 

2. Date of Test 

3. Test Level 

4. Student Name 

5. Student Nickname 

6. Class Code 

7. Test Administrator Code 

8. Responses for Each Test Item 



Behavioral Oblectivos Performance Ftl<^ 

The purpose of this file Is to accumulate student performance data on the behavioral 
objectives. This file is maintained in connection with the field testing of the 
objectives and Is Independent of the general data base system. Its application 
Is In the analysis of three general research questions as follows: 1) What ob- 
jectives are being used in the field testing? 2) I/hat is the relationship be- 
tween the teacher prediction of student performance on the objective, the pre- 
testing of the objective, and the post testing of the objective following In- 
struction? 
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3) Uhat i6 tho relationship between student perfomanoe on each objective and 
student mental age? 

Because ot the large number o£ objectives in ii^ld testingi only performance on 
objectives In the areas of Readlngi Wrltlngi and Arithmetic are being accumulated 
in the file* The file Is organisted by studet)":: and objective within class and 
contains the following principal elements s 

U Classroom Code (if applicable) 

2« Teacher Code (if applicable) 

3« Number of Students 

4« Student Code (if applicable) 

5* Date of Birth 

6« Date of IQ Test 

7« IQ Score 

8« Objective Code 

9« Content Outline Identification (where applicable) 

10« Grade Level 

111 Date of Pre-test on Objective 

12* Prediction, Pre-testi Post test scores for Each Student on the Objective 
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APPENDIX 



DATA-COLLECTION FOWIS 
and 

INSTRUCTIONS 



!• Fom0 Control Instructions 

2« Form 4 (Class end Teachor) 

3« Form lA blank 

4* Form lA review 

5f Form IB blank 

6« Financial Data 

7f OSAI Scoring Sheet (4 levels) 

8, Form 2 (Behavioral Objective Field Teat) 
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POfiMS CONTROL FOR THE MENTOR TITIB III PROJECT 



Iftttniotioru 

Follov the initial aequdnce given beloWi atepa 1 through 6, to eet up each olaaa 
in your aatellite area# In the Hall of 1972 i all of your olaaaee can be pro- 
ceeaed as a groupi After the fall of 1972 i each class should be processed as 
soon as it enters the program evaluation* Steps 1 through 6 require only one 
fom for each class and only one form for each teacher* To sio^llfy forms 
bandiingi the class and teacher forms are on the same sheet of paperi If there 
is more than one teacher for a given class i use additional teacher description 
forms and describe the class completely on these additional forms* 

As new classes and teachers enter the program evaluation, follow the six steps 
in the initial sequence given below* If a teacher or class exits from the 
program evaluationi send a note to Battelle indicating 1) the class and teacher 
codes that Battelle assignedi 2) the district* building^ and teacher names, 
3) the exit date, and 4) the reason for exiting* Cases of mass student transfers, 
major changes in model designation, and other unusual, though possible, changes 
can be processed with the data base system. Please contact Battelle directly 
for the procedures required in these cases • 

Steps 6 and 7 constitute an iterative updating sequence for each classroom in 
your satellite* This sequence has several major advantages over alternative 
approaches I 

a* Bach data element (name, score, etc) is entered only once and 
Battelle returns a new copy with the updated information* 

b* Changea in any data element can be accomplished at any time by 
crossing out the old value, entering the new value, and sending 
the form to Battelle* 

Ot Incomplete data can be completed at any time and sent to Battelle, 
allowing for entering the results of makeup testing • 

d* It is not necessary for you to retain any historical or archival 
information^ you will always have the moat up«*to«*date copy of all 
accumulated information* 

If you have any questions about the forms or forms handling, please do not 
hesitate to call the project data manager at (614) 299-3151| Extension 2452, 

Initial Sequence for Each Teacher«Class 

It Bach satellite will receive 30 blank copies of Form 4 (class and teacher 
information.) 

2* Bach sntellito will soo to the completion of a Form 4 for each teacher 
class and return to Battollo* 

3« Each satellite will receive a^ Form lA (Student Description) prepared for each 
class Identified on the Form 4U* This Form lA will contain the class/model 
identification, teacher identification, and student codes • 
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4» Baoh satellite will aee to the completion of each Form lA (student nauei 
entry date, blrthdate, etc) and return the Fottn lA to Battelle* 

5* Bach satellite will receive two forms, Form lA and Form 1B» for each class* 

at Form lA will contain the completed student descriptions for the 
satellite records. This form Is then to be used to indicate new 
entering students and exiting students* 

b$ Fom IB (Student Performance) will contain the names of students 
identified as being enrolled in each class and will contain spaces 
for entering student performance data* 

Iterative Uodatinp Sequence 

6* Bach satellite will periodically review Form lA (Student Description) for 
student drops and adds. 

a# As frequently as possible, Form lA with indicated changes should be 
sent to Battelle* 

b* Battello will return an updated Form lA with additional student codes* 

7* Each satellite will see to the completion of Form IB (Student Performance)* 

a* After data entry, Form IB should be sent to Battelle* 

b* Battelle will return an updated Form IB for the satellite' 3 review 

and a new Form IB with spaces for the next performance data collection* 

Additional Guidoltnes 

1* Battelle will assign all class, teacher, and student codes* Once a 
code has been assigned, that code will always be associated with the 
particular class, teacher, or student* You will be hotified of the 
class and teacher codes upon Battelle 's receipt of Form 4* Specific 
student codes will be Indicated on the Form lA and IB which will be 
sent to you. Additional spaces and codes will also be provided for 
students entering without a code. 

2, Always use Form lA as the master enrollment list* As students enter 
and exit, record this on Form lA and send it to Battelle as frequently 
as possible* The currency of the Form IB (Student Performance) will 
depend on your maintaining an accurate record of student enrollment 
using Form 1A« 

3* The term '^class" refers to one or more students under a single teaching 
model* Class does not necessarily refer to a physical location within 
a school building* 
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Form AA • Class Information 



Please £111 In this section 
for each model class in the 
satellite area. 



Date of Class Entry into Program / / 

Model Types Code No. Descriptio n 

School District Count y 

Building Nam e Level: P I J S 



No. of EMR Teachers In this class 



No* of Students in this class (est.) 



Form 4B • Teacher Information 



Please fill in this section 
for each teacher in the model 
class identified above. 



Date of Teacher Entr^ 'nto this Class 
Teacher Name 



Last 



DatQ of Birth / / 



First 



Middle 



Sex 



Race 



Teaching Experience in this type of model 
Total Teaching Experience with EMR's 
Other Teaching Experience 



years 


months 


years 


months 


years 


months 



Highest Degree (oni*^^®): none BA MA PhD EDD 
Major Area of Study (4Sg^*^) : ^1^,' * II?' other 

Minor Arefl of Study (*^one^®) : ^IS?* other 
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ESEA TITLE III 
PROGRAM MODELS FOR EMR STUDENTS 
FINANCIAL INFORMATION 



ClassrooM Cod e Date Prepared / / (1-11, "I") 

Butldlng Name Areat LC SC U R (13) 

District Nfltn e B attel le; S £ (14-27) 

General Information Total EMR 

ADM School District , , (28-39) 

Students School Building , , ^ ^ (40-47) 

EMR Classroom (48-53) 

Total School District ^ (54-59) 

Classrooms School Building _ (60-65) 

Financial Summary (dup 1-U, "2") 

Line 1 Administration , (13-18) 

2 Staff Support , (19-24) 

3 Supervisor of Instruction , (25-30) 

4 Direct Instruction ^ , „ (31-36) 

5 Auxiliary Services , (37-42) 

6 Operation and Maintenance (43-48) 

Total Model Expenditures , (49-54) 

ADM of EMR Students , (55-58) 

Average Cost per EMR , (59-62) 
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Ob,1ectlve IIC laplement the evaluation doslgn by S^ptecibort 1972 1 



Throughout this entire project » the Coordinating Center and the satellite centerft 
have worked closely with the Ohio Division of Special Education In Implementing 
and coordinating the objectives for this project. The development of the models 
and the Implementation of the creation of new classrooms representing those 
models become a very Important part of the project during the spring of 1972i 
Although it was originally anticipated that the project could select from class- 
rooms representing various models within the State school systems, it was 
recognized during the winter of 1972 that such classes did not, in fact, exist 
in any organized or coordinated basis. Thus, the spring and summer of 1972 
became a time when the development of guidelines and the development of forms to 
accept proposals to begin experimental classrooms in Ohio began. The project 
satellite coordinators worked very closely with the EMR section of the Division 
of Special Education on the development of the forms used to originate those 
original models. The Title Coordinators then worked with local EMR supervisors 
and other local staff personnel to assist them in the development of the proposals 
for the creation of those models. This process continued from May, 1972 until 
the actual creation of those classes in September of 1972. Because those classes 
were not on-going in the State during the first two years of the project, the 
actual pro-testing of classroom models could not meet the time deadline as 
anticipated in the original proposal. Recorded in Appendix A are copies of the ' 
guidelines for the development of models and criteria* for the establishment of 
nine different experimental models and an applicatlcn form for that original 
set of proposals. 

As Title III Coordinators worked with local people in the development of those 
classrooms, a great deal of time was spent helping write the proposal and 
helping structure the various models. These proposals were submitted to the 
Division of Special Education through the Title III Satellite Centers, and the 
State-wide Coordinating Center in Mentor. Project personnel were involved in the 
creation, adoption, and the development of these models. This rep.resents the 
major unanticipated outcome of the project in that the development of these 
models consumed a great deal of project time that was not anticipated originally 
in the proposal. As is Indicated in the report from Battollo, the Ohio Special 
Achievement Inventory (OSAl) was ready for pre-testing students in September 
and October of 1972. The actual classrooms and students Involved in those 
various models were not in operation, in most cases, until October, November, 
and December of 1972. Thus, the pre-testing was on a staggered basis and 
consumed, in many cases, the entire fall per model erea although each particular 
classroom was tested within a shorter period of time. The project continued to 
accept classrooms within the vaj.*lous models as late as the last month of the 
project, January, 1973. Pre-testing was also continued into January of 1973. 
Thus, the pre-test scores reported in Table V represent staggered input rather 
than a clear and neat one month interim as was orlglna.lly anticipated. Further 
refinement on the development of guidelines for the models and solicitation of 
additional models for the coming years continued to be part of the role and 
function of the Title IXI Satellite Coordinators. However, the objective at 
that point was to transfer that role to the local EMR supervisor and the local 
administration of particular school districts who were creating the classrooms 
within each experimental model. As indicated in Table III, there were as of 
January 31, 1973, 99 classes involved in the various models. 
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Ob^leotlye IIP Begin the evaluation of EMR models dvirlng September | 1972 « 
January I 1973. 

Ab indicated in the prevloua objective i the actual claadroono began anyvhere from 
September through January > 1972-1973 • Pre-teet scores wore taken on using the 
Ohio Special Achievement Inventory and the reading and arithmetic sections of the 
Metropolitan Achievement Tests. The results of those tests, measuring l,20l4 
students, are in the Coordinating Center. Sample copies of the print out sheets 
from Battelle are Included in Appendix A. The evaluation design calls for post 
test scores to be gathered in May of 1973, but because of the staggered pre-test 
dates and the short amount of time interim between, in some cases January and May, 
the probability of significant gain scores of any students is decreased. Hovevov, 
dates of pre-testing, interim periods between pre and post testing have all been 
recorded sM the statistical analysis will be attempted in the next project. 

Table IV lists indicate some of the basic statistical data of those first pre- 
test scores. The I.Q. scores recorded represent individually administered tests. 
Although there are over 775 "total I.Q." scores indicated, there were more than 
1,200 students in the model classes. This means that there were many students 
whose scores were not recorded. It seems Important that there is only one 
statistically significantly different mean I*Q. score for the various models* 
This occurs in the Verbal I*Q. mean scores between the selected educational model 
and the learning center. The small number of selected educational model 
students (6) makes this statistically significant finding less meaningful and 
thus is interpreted as a sampling artifact rather than one repi^esentlng a true 
discrlptor of the model. 

Further analysis of l.Q, scores per type of model at each level (primary. 
Intermediate, junior, senior) also showed no significantly different mean I.Q* 
scores. This indicates that students were placed in particular models independent 
of their I,Q, score. Although there were not statistically significant differences, 
there was a pattern of I.Q, scores to higher for the self contained classes than 
the learning center at the elementary level and lower than learning center at 
the secondary level* 

The results of the pre-testing on the Ohio Special Achievement Inventory (OSAl) 
are recorded in Table V, The OSAI Instrument is described in detail in 
Objective IIA of this report. Raw scores of student responses to the pre-test 
were standardized for each of the ih currlcultun areas. Test results for each of 
the standardized scores were then listed for all students at each level (primary, 
Intermediate, Junior high and senior high) for seven different models. These 
standard scores (Z) were then transformed by the following formula to eliminate 
negative scores and increase the total amount: 

transformed score » Z (lO) + 50 
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TABLE ;V 

MEAN I.Q. SCORES (PERFOBMANGB) 

MODBL 3? ^ N 

I* Solf -Contained 72.917 8.490 24 

2. Modified Self-Contained 78.500 9.847 18 

3. Selected Academic 76.294 8.528 17 

4. Selected Educational 80.667 7.367 6 

5. Half-time placement 74.667 4.619 3 

6. Learning Center 76.522 12.158 69 



MEAN I.Q. SCORES (VERBAL) 

MODEL X N 

1. Self-contained 75.250 7.719 24 

2. Modified Self -Contained 73.087 6.149 23 

3. Selected Academic 74.941 9.344 17 
4* Selected Educational 80>000 3.347 6 
5. Half-time Placement 75.000 3.317 5 
6« Learning Center 72.786 6.747 84 



HEi\N I.Q. SCORES (TOTAL) 

MODEL X V^' N 

1. Self-contained 71.031 7.624 64 

2. Modified Self-Contalned 71.323 7.473 96 

3. Selected Academic 71.526 7.158 78 

4. Selected Educational 73.083 6.762 12 

5. Half-time Placement 70.200 6.215 10 

6. Learning Center 71. 163 7.735 264 
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Curriculum Areas wet 



1# Learning to Appreciate > Cttaate, and Enjoy Beauty 

2* Leerning to bi a Responsible Citizen 

3. Learning to Conaunloate Ideas Through Arithmetic 

h. Learning to Corni\unlcate Ideas Tlirough Reading 

5. Learning to Cottj'Aunloate Ideas (Verbally and in Writing) 

6i Learning to Eavn a Living 

7. Learning Homeaiaking and Family Skills 

8. Learning to Keep Healthy 

9. Learning to Live Safely 
10. Learning to J^anage Money 

11 • Learning to Travel and Move About 

12. Learning to Understand One's Self and to Get Along With Others 

13, Learning to Understand the PhyslcaJ. Environment 
ih. Learning to use Leisure Time Widely 

Models are; 

1. Self Contained 

2. Modified Self Contained 
3* Selected Academic 

^. Selected Educational 
Half •Time Placement 

6* Learning Center ' 

8. Mainstream 



The large number of these scones makes single interpretation very difficult! In 
eneral. scores are statistically significantly different from each other 
P> •05) if the number of students in each group is over 30 and the numerical 
difference is greater than three. In those cases where the N of either group is 
below 30 ♦ the numerical difference must approach four or five to reach signifi- 
cance* Statistically, significantly different scores indicate that the average 
responses for a particular group differs from the average response of another 
group to such an extent that that would happen only five times out of 100 by 
chance alone. 

The following general observatioJis can be made from this descriptive dataj 

1. Students at the primary level self contained class achieve at the same 
level as students in tha learning centers* 

2. Students in the half time model achieve more poorly than other models. 
However, the 2k students i^epreeented there are from only two classes. 
This probably reflects a sample difference rather than a model difference. 

3. Students at the intermediate levi^l self contained class perform below 
students in the learning center In several curriculum areas. 

I*. Students at the Junior high level in the self contained class achieve 
as well as or higher than those in the learning center. 

5. Students at the junior high selected educational placement model achieve 
above other students. 

6. Students at the senior high level self contained class achieve as well 
as the learning center students. 
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Qoal III I nstitute In the State'-vlde EMR program during the duration of this 
pr^^^ect an on^'going evaluation Drocess vhicti is based on behavioral o'jijeotives 
and vhich vlll continue at the termination of the proJeot . 

Objective III A Refine the evaluation design and instruments with selected 
evaluation toaohere* 

Throughout the entire tvo years of this project^ there has been a process 
evaluation component which has included teachers » EMR supervisors, and our 
Title III Coordinators in the design of the field testing of ev6ry step of the 
project* The field test forms, which we are presently using, have been refined 
at least three times, copies of those can be found in Appendix k. Teachers 
using the field test forms pick up several different kinds of Information 
on each EMR student.. The form required a prediction from the teacher on the 
success or non-success of the student on a particular behavioral objective. 
T-1 (test 1), means that the teacher actually pre-testa a students If a student 
fails, she then teaches the student that particular objective and tests him the 
second time (T-2), If the student falls at that point, the teacher can re-assess 
and decide whether or not the objective was relevant or whether she should attempt 
to continue with that objective or attempt another one. T-3 on the form asks 
the teacher to come back 30 days after the T-2 test to again test the student on 
that same objective. For students who pass T-2, this is e 30 day check on the 
retention of that fact. Form number 2 for the field testing is a form which 
gives information regarding anecdotal comments a teacher might have about a 
particular objective, the teaching strategies and/or the materials she used in 
working toward that objective then a listing of the kinds of akills that the 
teacher needed to teach that objective. The form 2, the anecdotal sheets, 
throughout the project were evaluated by local Title III Coordinators end in-put 
back into the objective, the change in format of the objective was given to the 
objective refinement committees during the spring of 1972, the summer of 1972, 
and the fall of 1972 when those objectives were being revised. 

Another part of the refinement of the evaluation design was a series of pllo't 
studies using each of the instnunents that ore used to record student progress. 
The first year evaluation has a series of those pilot tests. A pilot test was 
also done this year on the Multi-Dimensional Attitude Scale of Mental Retardation. 
The results of the study (see Appendix B) indicate that attitudes of people 
do differ as they have contact with a mentally retarded child. The study ' 
was done in Warren, Ohio, with teachers of educable mentally retarded children 
and teachers of normal children. Therefore, the refinement of the evaluation 
design was done basically by selecting instruments that have been used in the 
past and have been successful with gaining information on these students and then 
piloting those instruments to make sm^e that not only can the computer handle 
the data, but that the d^/ca that is gathered is relevant. 

Another part of the evaluation refinement relative to the products of this project 
was the very elaborate system of recording, selecting and preparing objectives. 

As this product was created, certain uniform definitions were created and 
taught to all participating teachers. Each objective written went through a 
process of revision and investigation outlined in Figure 1. 
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Yet) another very meaningful part oi the procese evaluation for thie project vae 
the bringing together of all coordinatora and IRC representatives from each 
satellite area on a monthly basis* These Advisory Board Moetinge vere conducted 
monthly throughout the two yeata of the project* Minutes of those meetings can 
be found in Appendix A« It wad during this kind of continuous input and feedback 
from the local area that managerial decisions could be made and focus of the 
project could be changed when deemed appropriato» A close investigation of those 
minutes will reveal that a great deal of business was done and that input from 
the grass-roote level did hav6 an affect and change in this process. 

Objective Ii;iB A ssess pupil gain of teache rs vho ^ha y e been invoW^^ versus 
students of teachers not ^t^volyed f ^ 

The intent of this objective was to use a control group of teachers and students 
to look at student gain of teachers not Involved in the project. It became 
apparent toward the end of the first year of the project that this objective had 
little chance of being carried out* Because of the regions within which each 
coordinator was working had a limited number of EMR teachers, and because we asked 
the coordinators to select the teachers for the first part of the project for the 
training sessions who were the most responsive and could provide the best input 
to the product, we had from the beginning i a selected group of teachers who were 
the CKpetlmental teachers* This meant that there was a selected group of teachors 
who were the control teachers* This, in itself, would bias the result of any 
kind of experimental-control study* In addition to that, we found that even the 
control teachers (those teachers not involved in the project), many times had 
course work and/or experience in behavioral objectives* In the duration of the 
two years, behavioral objectives became very predominant and a very popular 
Item of instruction and thus we really could find no teacher who was purely non- 
involved with behavioral objectives. Therefore, a control group of teachers 
whose background ve really did not have control over presented another bias* The 
third reason that this objective was not met is that the large number of students 
who were tested as a result of the experimental teachers took a disproportion in 
the amount of tester time and project dollars and there was not the time needed 
to test control students deemed by the director, priority time, or dollar 
expenditure and, therefore, the control children and their teachers were not used* 

This is the one single objective that was stated in the proposal which was not 
met at all and because of the above stated reasons and the difficulty of pre/ 
post data collection, it remains an unmet objective at the end of the two-*year 
project. 

Objective IIIC Begin to Disseminate information on the behavioral objective s 
cheated. 

Oblectlve HID B egin to disseminate Information on evaluation based on behavioral 
objectives * 

Throughout this entire two-year period, the behavioral objectives and the creation 
of them has, in fact, been part of the dissemination process* Because so many 
teachers were Involved in creating the objectives, they had at their fingertips 
in the work sessions for the entire two-year period, those lists of objectives 
that they and their satellite area had created* In February of the fii*st year, the 
slide presentation with the cassette tape narration was presented to every 
satellite area for the purpose of explaining the project to that region* In 
addition to that, the first spring a small, one*page, three^fold brochure was 
created and throughout the two years, we have disserainafced over 3,000. It is 
presently in its fifth printing. Of course, the proposal Itself was printed, 
distributed in every region in the State involved in the project and hundreds 
of copies of a brief 15 page description of t>:e project was distributed In each 
^ and to all people inquiring for their information. 
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During the process of refining the objectives as they vent through the fourth » 
flfthi or sixth drafti they were refined to their present format* Four hundred 
sets of IS books were printed through the Coordinating Center in Mentor » Ohio« 
Those 400 sets have had limited distribution within the State of Ohio, and a 
dletrlbution list is indicated in Appendix C. That list Includes a cotoplete 
set of objectives to every teacher in a model classroom, a number of sets of 
objectives for each coordinator to use in teacher training sessions in satellite 
regions I numbers of copies and numbers of sets were distributed to State 
department personnel, Co the nine major universities in the State, and to the 
remaining seven instruotional resource center programs ef each regional special 
educational service center. In addition, a set was sent to the University of 
Michigan microfilm, the Bureau for the Education of the Handicapped in Washington, 
D«C., to the major national consultant, Dr« Edward Meyen« 

To accompany the behavioral objectives as they were sent to every Instructional 
resource center in the State, the Coordinating Center and selected regional 
Title III Ccorlinators developed a teacher training kit. This kit consists of 
a teacher-Instructor's manual which includes five two-hour module?* of Instruction 
on how to use behavioral objectives In the classroom. In addition, the kit 
contains sets of participant materials, selected reference materials and two 
film strips concerning the project and the use of behavioral objectives in the 
classroom. Filmstrip number one entitled, ''The Process of Change," talks about 
the intent of the Title III project and the creation of the behavioral objectives 
themselves • PUmstrlp number two entitled, "Teaching Strategies, The Field 
Test Process/' ia a cai^efuX description of the field test process including 
both pre-test and post testing and the field test model. These teacher training 
kits, 30 in number, were distributed' to each instructional resource center, to 
all nine major university teacher training programs, to selected State department 
personnel, each satellite coordinator, each satellite region and the Coordinating 
Center* There were a number of newspaper articles published, copies of selected 
samples of which are found in Appendix C» At least two articles have been 
submitted for publication concerning the project for those two years and others 
are in the developmental stage at this point < (See Appendix C). 

Dr. Edward Meyen. national consultant to the project, has addressed State-wide 
meetings at the end of each year of the project. A copy of his first year 
comments can be found in the First Year Evaluation, A copy of the outline of 
his second year comments is included in Appendix B. Of special Importance in 
the second year comments is a summary comparison of this Tttlo III project with 
three other EMR curriculum projects of national Importance. 

In December, 1972, ^he project fiscal agent of the Coordinating Center was 
requested by the Ohio Division of the Department of Education, office of Title 
III, to participate in a national validation study. This study conducted 
through the auspices of Title III at the federal level was an attempt to select 
from across the nation, those Title III projects which were effective in their 
region of being recognized in the State of Ohio as one of the top six Title III 
projects which, In itself, was an honor for the project and part of our 
dissemination process. The validation study Involved the filling out of a very 
elaborate data collection document by the project director, by the State 
Department of Education of Title III, and by a team of site visitors who visited 
with the project for two days in February. A copy of the evaluation report as 
made by those site visitors is found in Appendix C showing that the project was 
seen as a very effective project and worthy of national dissemination. 
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Objcotlv III B Conti nue teacher workshope with 120 teachers of clasees in the 
aattPle m^jirf^laT 

Throughout this project, the Title III Satellite Coordinators have been involved 
with training teachers of EMR students. In addition to that training, those 
aehool districts which submitted a proposal for an experimental model and were 
granted permission by the State Diviaion of Special Education to operate such a 
model had teachers of EMR students who were involved in In-service » As is 
indicated on the January, 1973 report froo Battelle» 99 such classes were involved 
at the termiaation of this projeof.. The majority of thaae olacaee, a« iadleated 
In Table III, are situated in modfile involving the self contained program, the 
adapted self contained program, t^ie learning center model. At that point, the 
mainstreamlng model and the half-day model were almost entirely without classes. 
These teachers were Involved in in-service education in the fall of 1072 and 
have been involved field testing the behavioral objectives during that fall and 
in the winter of 1973. Field test data in the curriculum areas of arithmetic, 
reading, and writing has been collected during the last helf of the first year 
and the entire aecond year. 

In addition to this in-service education and field testing, the models teachers 
were iwolved with assisting in the pre-testing of the Ohio Special Achievement 
Inventory and will continue with the post testing in the spring of 1973. 
Although originally it was stated that there would be 120 teachers, the number 
of requeate for experimental models did not reach that maximum, therefore, 
only 99 models were on-going. It ia anticipated in the future that additional 
classes would be included in this target population until we reach the maximum 
of 120 classes* 

Objective IV Tr aio EllR toachera in thrco j ma^or cttl oe of Ohio to write an d field 
toflt behavioral ob^ect iyea. 

Although five of the eight major cities in Ohio were Included in the original 
aatellite areas, the numbers of teachers, the Input from those major city areas 
was not proportional to the number of EMR students in those regions it\ the first 
year of the project. As a result, during the second year, an emphasis was 
placed on including the remaining three major cities as teacher input to the 
project. Table III indicates the numbers of teachers from Dayton, Columbus and 
Cleveland, Ohio, who were involved In the spring of 1972 and the fall of 1972. 
Each of these teachers received 10 to 15 hours of in-service education and 
field tested behavioral objectives from the project. During that field test 
process, teachers had opportunity for anecdotal comments on objectives and 
thus input into the rewriting of objectives before the final draft was printed. 
It could also be noted, at this point, that those three major cities did 
present proposals for experimental models (15 classes In Dayton, 2 In Cleveland 
and 2 In Columbus). Pre/post teau scores are also Indicated on Table II 
showing that teachers Involved In this In-service education did. Indeed, receive 
significantly higher scores upon completion of the In-servlce education pro- 
gramming. 
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E. DISSEMINATION 
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E DieSEMINATIOW 



Since the dleaemlnatlon activities of this project were written under a major 
objoctivo, the roeulto can bo found under Objective HID* 
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RECOMMENDATIONS 



p. RECOMMEKDATIOWS 



Thd results and conolualons of this project hove led to nany rocomnondatlons 
which have already been Itopletnented. Major among these vas the submittal of a 
proposal to E6M Title III which would carry out the recoTomendatlonSt That 
proposal has been approved i a project '*EMR Program Development*' has been funded j 
and is presently in operation carrying out the recommendations listed below* 
Many recommendations which came as a result of the elaborate process evaluation 
within the project are not listed* 

Major Recommendations t 

U The objectives created should be evaluated through field testing and 
revised as feedback Indicates* 

2* Teachers of EMR students who have multiple handicaps should be in« 
volved in the training sessions and revision process* 

3* The experimental models should continue to be evaluated using the 
design created in the first two years* 

4* An in-service education program should be established » utilizing 
the university staffs of the nine major teacher training programs 
lx\ Ohloi based on a performance-competency model* 

5* The coordination center-satellite center model should continue as a 
vehicle to Implement objectives in a State-wide project euch as this* 

A detailed analysis of the results of these recommendations Is available on re- 
quest from the Mentor Coordinating Center* This analysis is the proposal for the 
project '♦EMR Program Development." 
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Moffeinget) thomas L, 
Title ^ "Program Models for EHR Students^^ 



Mentor Exempted Village School District, Mentor » Ohio 
Sf^oneorlng Agency - Ohio ESBA Title III 
Report Noi 

Publication Date • May, 1973 
Grant No* - 45-71-207-2 
Note - 100 

Descriptors • Administration, Mental Retardation Curriculum, &rdliiatlon. 
In-service Education/ 

Identifiers * Critt^rion reference test. Behavioral objectives, Alternative 
models for EMR students^ 

This report includes a summary of the major findings of a tvo*year project 
involving eight regional satellite centers in Ohio. Major objectives were 
achieved In setting up a State*-vlde process of in-service education of EMR 
teachers I. In addition, behavioral objectives and teacher activities Vera 
created in 14 content areas representing the ^'Persisting Life Problems^V appironeh 
to EMR curriculum* Eight different educational models were identified iv4 <iX^$iM 
vere initiated in utban^ rural, and suburban areas of Ohio. An ac^i^MVeme^ 
based on the EMR curriculum was created and students in the ti(tbdel cla^scis were 
pre-tested« 

The following major recommendationi ire being car::ied out, at the present time, 
in another project* Results of thut project will be available at a later date} 

- 1« The objectives created should be evaluated tKr^rOgh field testing and 
revised as feedback Indicates. 

It Teachers of EMR students who have multiple handicaps should be in*' 
volved In the training sessions and revision process^ 

3« The experimental models should continue to be evaluated using the 
design created In the first two years « 

4« An in-service education program should be established, utllieing 
the university staffs of the nine major teacher training programs in 
Ohio, based on a performi^nce-compefcency model* 

S« The coordination center - satellite center model should continue as 
a vehicle to Implement objectives in a State-t*lde project such as 
this. 
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APPENDIX B 



Evalucitlon Conference 

Ohio Title Itl Project 

Columbus, Ohio * January 26 ^ 1973 



I* Introduction 

A» role in this project during the past couple of years has varied. 

1# Dissemination of views and biases t 

2» Reactor to oaterials and ideas. 

3* As a participant in conversations • 

4. As a rather unobtrusive observer of your activities • 
B» Ify preference is to function in an interaction role* 

1* I learn from the questions asked* 

2, )^ responses can be checked out on the spot* 

3« You communicate a great deal about your attitudes, concerns, and 
perceived reinforcements in the process of interaction* 

4» Interaction allows you to direct the discourse to areas of your 
concern in contrast to the areas I assume represent your concerns* 
C« The function which is exceedingly important but presents the most 

difficulty to me because of the nature of my involvement in the evalu-* 

atlon role* 

1* Evaluation implies the making of quality based Judgments* 

2* Evaluation allows for positive reinforcement but typically stimulates 

a concern for the negative aspects of one's work or product* 
3» Evaluative comments « if you will, are as valid and meaningful 

as the qualifications of the evaluator - You do most of your own 

evaluating then you need to be concerned with your evaluation skills 

and obligations t 

4. As a consultant who Is only Intermittently involved in your project, 
let me share with you my evaluation frame of reference so that we 
can reduce the risks of misinterpretation. 
a» I bring to this project, a sincere interest in curriculum 



development for the mentally retarded which in couched In a 
context of experience involving varied curriculum development 
activities.. 

b. My attitude towards the involvement of teachers in development 
and decision making regarding curriculum Issues is exceedingly 
high although realistic in that I recognize your history 

of limited support and opportunity to develop all of the skills 
essential to curriculum development. 

c. My relationship to this project has not been one of an evaluator 
with extensive involvement or access to data. Thus, I am 
working from a perspective which is Influenced by only a repre*^ 
sentatlve sampling of your progress. These indices are then 
measured against the expectations I hold for you and your project. 

5. This evaluative fraoie of reference obviously creates a bias towards 
identifying and supporting those positive aspects of your project 
which I heartily endorse* 

a. To focus only on the positive would be to reinforce the obvious 
and to risk stiffling progress towards a better product and 
more effective instruction for the handicapped. 

b« Let me acknowledge your productiveness as individuals and your 
collective progress. Let me also compliment your State agency 
for its support of such an undertaking and the Mentor Public 
Schools for their willingness to assume a task having logistic 
problems which would have been a challenge even to General 
Patton. 
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c. Let me theh proceed to direct my attention to eome erea^ of 
evAluatlon vhlch may vatrant your attention* I i^lll be 
focusing on concerns concerns vhlch In total barely tip 
the scales against the positive credits you have accrued, 
d* Ify approach vlll be to cormnent specifically vhere it Is 
appropriate and In other cases direct ydur attention to the 
areas deserving evaluation* 
e. Detailed comparison with other projects* 
II« Sources of Evaluation Evidence 

k. Page four of the grant proposal contains a statement against which X 
have tended to check say assessment of your progress* 

*'There is a need to develop a process of change in the educational 
milieu which will effectively change the behavior of teachers 
and students as quickly as research evidence Is available**^ 
1# Keywords: - change 

- educational milieu 

- behavior of teachers and students 

- evidence available 

2. Zn any discussion which is evaluation oriented^ it is essential 
to identify the evidence on which evaluative Judgments are made« 
B« Source of Evidence; There are two sources of evidence^ l»e> > the 
observable progress related to specified objectives and the less 
apparent but possibly most Important evidence which comes in the form 
of professional growth* 
Observable Evidence 

1* Product • Objectives 

a* The process of developing the objectives will likely prove to 
be more value than the objectives - return to this statement * 
h. The objectives serve several purposes beyond a basis for 
evaluating models* 

1* They collectively define the specifications for a curriculum* 

2* They serve to draw attention to curriculum areas typically 
overlooked, e*g*f art appreciation* 

3* They effectively communicate the iiqportance of incorporating 
the teaching of basic skills with. application skills* 

4* They provide teachers with the necessary curriculum support 
on which to push for materials and needed resources which 
are relevant to the kids they teach* 

5* They are highly visible and represent observable evidence 
of a curriculum effect in special education* 

6* They can serve as a very useful tool in the structuring 
of units of Instruction, l*e*, reorganising objectives 
according to topical areas* 

7* They can be used by groups of teachers In a building or 
system as the basis for curriculum planning* They could 
organise the objectives in parallel linear fashion to 
reflect the intended scope and sequence of their program by 
level* In such an activity, ihe objectives as discrete 
objectives become the focal point of the planning efforts* 
Decisions are made by the teachers regarding the validity 
and sequence of the objecMves* 

8* As individual and groups of objectives, they have tremendous 
use with the stations approach where selected Interest or 
topical related objectives can be related to needed 
materials and used as self directed Instruction* 

9* They serve as a mejor resource to teachers in designing 

evaluation programs to assess groups and individual progress. 



10* tkHf i^Tovldtt a means for 6ri«atlnk taaeharti adiftltii«ttatova» 
and paraQtA to ctitHteliAi aad to levala of «qiactatlon fo|r 
kida in a pattieular pt^rata* 
e# Coticorfli i« ihay continua to vary la apaoifiolty and In genatal ara 
too apaclflo* 

Iha apealflolty glya* you a falaa aanaa of oonfldanca« 
Tha user aaaumaa graat4t cotnplataoaaa than agists* 
Tha aaQuanea also Impltas an ordat which may not a)tlst# 
2# To ba of watiimirt usoi tha fieaohar muat ba aequalntad vlth 
all of tha objactlvaa vfaloh v%ta talavant to hat group* It 
would alao ba advantagaoua If aha wara knowladgaabla of tha 
objaotlvas which are relevant to tha group abova and below 
here* In tha present form» it is a laborious task for a 
teacher to obtain such familiarity with the objectives. 
Could you design a fold out chart which prasants the basic 
diiuaasiona of the curriculum in a visual manner • add to 
outline t 

3. Teacher will need help in reorganisiat objectives for 

particular purposes* fixamplefi of unit organlaatlon or for 
remediation on a particular problen might encourage mora 
use of thefD» 

4« Entry into the objactlves is not difficult* given that a 
teacher has a reason for using thm^ could you provide a 
manual which sites examples and of uses and procedures to 
illustrate how the objectives were used* 
2. Structure aa Evidence 
a« Low attrition rate 
b» Maintaining reasonable schedule 
c. Testing procedures designed and data being retrieved 
d« Your organiaatlon has allowed you to meet moat deadlines and 

to produce your product goal. 
Aooarent but Vndocumentad Evidence 
1. New skilla 
a« teacher 

1* writing objectives 
2# working on curriculum team 
3* curriculum organisation 
4» integration skills 

5* negotiation - hopefully iu raachiog agreement on 
objections 
t. Attitudes 

a« toward tha project 

b« toward objectives 

c# toward curriculum development activity 

d« toward future iovestment 

e# toward evaluating curriculum 

f • toward evaluating project performance against curriculum 
objectives 

III* Comparison of Ohio Project agalnat one other project as a means of sharpening 
your perception of where you are in terms of evaluation * 

B*S*C*S« Transparencies 
IV* Cca^arison of Ohio Project status 
Yeshiva 
Connecticut 
BSCS 



Cooipatlsmi pvMentad to remind you o£ othev ptojectfl 

- fltlaulate you to think of how you will u&e them 

« edoourage you to mako eoisparisoiid and gain from the evaluation effotta 
of othera 

- give yott confidence in your achievement a 
Suanary 

1« You have made groat atrldea# 

2« You have a product \fith capabilities^ 

3» Your product can not a tend alone* 

A« You need to inveat in making it uaeable give teachera a purpoae 
for uae# 

5« You hove demonatrated that a large field based'teacher oriented 
model can work that in itself la an achievement* 
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Model Process 



Ohio 


Behavioral Objectives 

Keed Area 
Level 
Terminal 
IdBtruotlonal 

Content Analysis 


Gurricultim ^ Secondary Goal 
Evaluation • Prl^nary Coal 
Multl Level 

Field Based Production 
Evolving Skills 
Central Control 
Ohio Based 

State Agency Accountability 


Yeahlva 


Eacpandliig Concet^tual Hodel 

■ 

Inductive Teaching 
Cognitive Levels 


Curriculum Product Goal 
Internal Decisions 
Justification^ Base 
Development Hodel 
Staff Team 
Assigned Tasks 
Beavy Field Test 
Content Oriented 


Conaectlcut 


Interact j!:ve Hodel 
teacher^pupllMsaterial 


Curriculum Product Goal 
Math for EHR Plus 

Central Staff Production 

Subject Matter Influence 

Design * Hithod * Skills • 

Activities • Assess 


BSCS 


Ecological Model 

Inquiry Strategy 

Cognitive Levels 

4 Appllcatlonal 
3 Relational 
2 Conceptual 
1 Perceptual 


[Xirrlculum Product Goal 

Subject Matter Influence 

Learner Assessment 

renting Team Oriented 
[{uallty Froduct Oriented 
Strong Formative Process 
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staff Field Teot 



E 


Ohio 


Teachor tMk Force 
Adsalnlstrative Core 

Coordioatloa Force 
Cooeultanta 


Ohio (only) 

Across Models 

Mo Content Control 

Mo Teacher Contrast Groups 

Taxget Cbjectlve Mot Currlculutn 

Fotmatlvd only ' 
Pupil Data 


Yeshlva 


Center Concept 

Dev^lopoent Core 
Related Unite 

Bvluatlcm 

Media 

Reeearch 

No Subject Hatter 

Speolallet 
Field Feedback 


Matlonvlde 

Planned & Fortuitous 
Large Scale 

Major Coordination Thrust 

Field Test/Intj»rpret/ttesearch 
Cycle 

Formative & Sunmatlve 

Crlterlra Measures 
Teacher Feedback 


Gonnectleut 


Development Core 
Special Ed type 

Subject Matter 
Specialist 

Fotmatlve use of 
Consultants 

Media 


Regional (selective) 
Assumed Representativeness 
Central Coordination 

Valred Arrangements 

Formal Feedback 
Toting 

Formative Bsphasls 


BSCS 

\ 


Curriculum Team (small) 
l^ec* Ed# 

Bvaiuation 

SdLeuce 
Writing team (heavy) 

Advisory (limited) 


National Pockets 

Formative Bsphasls Plus 
Sunnative 

Central Coordination 

Evaluation of Material and 
Products Prior to Publishing 





Fbntitt Cott (tiotltUt L 


EF 


Ohio 


Beiu^rioral Objective^ 
Issued Gont^ 
MultlpU Batty 

TMCher Conttol 
Catd Syttw 
Limited AatlvltlM 

Individual 


Teacher ftork Costs 
Coordination 

Product Costs 


Yeshiva 

- 


LinMt Design 

Detailed Teacher 
Direction 

Pupil Materials 

Social Focus ^ 
broadly based 

Veacblttg Method 
Influences 

Group ^ 


Central Staff 
Field Test 
Evaluation 

Independent Resources 


Connecticut 


Skill Oriented 
Activity , Structure 
Laboratory Kits 
Multi Media 
Multiple Entry 
Open Ended 
Strands 
Individual 
Language Control 


Central Staff 
Activity Product Costs 

'-'■^ 
<■ ■ 


BSCS 

uc 


Package 
Teacher Manual 

Activities 
* Bes^mrco 

Content 
Evaluation 

Multiple Entry 

Oroup. 


writing Teams 
Media Production 

Central Staff 
Purchase of Resources 



Obi«> 


Te«eh6r Skill 

Fl«ld ¥Mt Ineoopldte 

T«aehe«f Ed, Plana 


Hd^uired teacher Knowlodga 
Hanagaoent for Uaa 

Falae Confldenoa 


Yeahlva 


"saf" itt print 
Family - Field Teat 
Secondary • Plan 
BIPP « Plan 

Edtabllahed Center 

> 


Entry 
Scope 

Curriculum Intluence 
Group 


Coimeetlettt 


Mumbere ^aratlona-Sete 
In Field ¥eet 

All Develepinont 
Eccept Fractions 

Laboratorlea Hearing 
Completion 


Time Lag 

Teacher Knowledge 
Management 

Quality Control Data 


BSCS 


He Now « Upper Blem« 
Print 

Me & My Eovlroment - 
Junior High • Field 
Test 

Me Aa an Adult - Study 

Pre-Me-Kow - Study 
Loifer Elementary 


Integration 
Teacher Acc^tance 
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kMrM*^ \ ■ ' ■: ' . dido fioii^ 

ColilKubuil OWd • i$Ud$x^ 26 1 1973 

I* tnt«6dii6tt6ti 

Ai M^tmiKig tiie £ut^^« d£ i ptojMfc dU6h m this Itivdlvei a eertaiii aaoiitit 

B# tout fatidite is «lto gtMtly InfluMseed by your track record » 
tcliehori you vorked vlth 

2« nchool dlstrlctB 

3. perceived value of your product 

4* Stete agency eeeeeemeat 
0* Your future le also influeiided by jrour ability to ai^ly vhat ytMi learned « 

l# new ekllle*teactiera«»coordinatord 

2* logistic problema 

3* selection of people^ l.e^ matching talent vith task 

4* reward systeoa ^t seemed to provide you the least return? 

5# vbat payoff from consultants? 

6# reasonable goal setting <• Did you take on overly ambitious tasks 

last yemr? 
7* What tasks were the iMst difficult? 
g* Where did you fall ehort on resoi\rces? 

9* Did la/)t yearU activities uncover needs related to general goals 
of the project? If so, can you operatlonallse these into objectives? 
II. Conversation with I>r« Moff singer 

A. A basic (tuestlon emerged relative to the future » l»e#, ^*do we haire a 
curriculum?** 

1* In its present form, the collection of objectives makes too many 
assumptions to be considered a curriculum with substance, 
a* Assumes teachers know how to use the objectives in planning 
their program. 

b. Assumes that teachers can recognise the inconsistencies in 
specificity and can make appropriate Judgs^uts In establishing 
a balance in the curriculum. 

c. Assumes that teachers haire the skill to reorganise the objectives 
according to a strategy from which they can teach and in turn 
select or develop materials, e.g., units, core areas, stations, 
etc. 

d. A substantive rationale for the designer, selection of areas 
and lnterrelatednes4 of content is missing. In other words, 

a Ju$tificatlon for the design axvd content is needed aloog with 
guidelines for application In classroooos and programs, 
e* At best, it represents a curriculum only for a few select 
teachers who uoderstand tlie development process the Intent- 
and who are sufficl^tly knowledgeable to make the transition 
from objectives to organisation for her situation, to planning 
for kidSf to teaching. 
B« the planned goals discussed Included i 

• enhancing the use of the objectives by all Ohio teachers 

• implications for teacher education 

• broadening development goals to Include readiness and vocational 
skills 
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fahtnbing 1110 of objtetlved 

a* Modila 6f ^fffttnt uses Is essentUl - guldelioM «rft not 
iaough / 
1* tmits tttlfi$ objoofcivoft M outline 
2* toaeh^ nad« t08t« ba0«d on objootlvoo 
3* hita to t««oh attlllo «nd Inforo&tion vblch uaoA objootlvoa 

t!i« •poolfioAtionn 
4» •tatloM - vbotfo obJootlifM ar« rolttod to aotlvitioo with 

••U oonnoetlng orltorion 
5* oo4«l eU<fltoe*Bi to 1>* irltitod uSioto tOiOlio« !• on^loylng 

th(ft.ot>Jootlyoi'''v 

6. voporting (pttj^ll ptogrooi) piroo«dtt«oi bM«4 oa ob joetivoi 
7« toactMur imUniag $«ttoa to b^ tltod in in«t«i;vioo tr«iiila$ 
8* paokagod audio tapfd pvograBia to taaob tfaobata to #rita A 

objaotlvaa aad to atilttaU tbii iya^^ tbair iaa^^a 
9. anaedotal aoooimta oi hbir ta^tiaifa tuv/a Mad than titb 
provlaion £ot: pariooal eontaOt:;':^ 

10. eaapla natatiala llsta «hieit ata baaad on aalaetad oblaetivai 

11. eunrieulutti plana ubioh ha<ira bacn vjeonatntotad and daalgtiad 
to fit a local pvoetram or olaas 

b* Naad an ln>-8ar^a ayacan vbAohi 

1* aaauraa diataainatioa of ittfoxmation 

aceaaa to objaotivaa 
3. aeoeao to insttuetion 
4« aecaaa to aiodala 
S* ti9a taachar aa inatvuetor 
6* noduia davalopQMAt 

7. iuelvda opti<ma for taaebar laaming 
Dnplioationa for taacbar adttoatiott. 

a* You bava a raaourca to of far taachar adiication «> training baia • 

aoda data ^ akiUa . ' 
b« Tha naad for pra*aarvica prograua to ineorporata trainii^ 

applicabla to tha projaot ia aaaantial 
o« Naad to datenaina tha ooiapatanoiaa rattuirad to? 

t« undaratand tha objaetivai 

2, to taach than 

3* to construet than 

4* to tranelata thorn to taachjng atratagiaa 
d, ihe coapatanoiaa will ba 90% ganario* 
a. Naad, to coma up with an inotruotlooal nodal ifhicht 

- haa <tuftllt7 control over content 

- la auffielently vlabla to allov for ravlslon 

- provldaa altamatlvaa for student behavior 

- allows for aalaotad involvenant of professors and classroon 
taechars In training 

• will sustain itself without project support and nonitoriog 

• draws upon field talent as wall aa professor talent in 
developing axperieneea 

- allows for skill attainnent «a wall as satiafjring of degree 
requircnents 

- is autually acceptable to participating teacher training 
inatitutiona 



3» .IVtttvti>i«)» •.tft^ltMiQ^ Vocational ■ "::x:5ls» 



«* both at* high i^rlority natidttally 

oaMftd <;b aiaaaa youv 6Wa tf*a^ ^oif «sipaiiMioa 

• yottt taakd ralatod t» ia^t&TUii tlia |NWikasa tad taaehiv 
ttaloing la ettougtt to tait yottit Attatgy aad v««batr«e« 
e* mot fu> expanding tha objaotlvaa in aithat ditaetion» It ia 
tn^mnt fot you to litat ktm ^Amt >dii ham^a la tha pratant 
paekaga* Yon naad to aoaplata tha Hald taat • tita out bow 
uaaabla tha objeotlvaa aifa. 
d« Civao pmaa valua and validity » yoa vould naod td $itg«BB tha 
eoat affaotlvanMa of pireduolng objaotlvaa thr<»«h tf^ 
taana* With fairar taachaya of tha praaohool and voaatl^l 
lcfvala» aoothat prooaaa may ba mora a£fe<itlva» 
A» Ihra-Sohool *• liBpUo&tlona 
5» Vocational • iaplioatiooa 
6. Araa wotth conaidorlng wfaon ifaady 
• aaverely handiottppod 
» high priority 

- objeotivo format oost appr<vriate 
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APPENDIX B 
PHOT STUDY WITH THE MASMR 



The Kultldiwenslonal Attitude Scale on Mental Retardation (MASMR) has been 
developed to assess the attitude of people toward tuental retardation In five 
major areas.* Those areas and a brief description of the interpretation of 
scores in those areas follows* 

1* tnteRration-^Segtegatlon (INSR) 

The higher scores on this subtest indicate that the person taking the 
test favors integrating retarded children into regular classes* Lower 
scores indicate that the person favors placement of retarded children 
in special classes* 

2» Overfavorableness (OVER) 

The higher scores on this subtest Indicate a willingness of the respondent 
to attribute overfavorable characteristics to the retardate* 

3. Social Distance (SDIS) 

The higher scores on this subtest assume that the respondent does not 
mind recognizing} living near> or being associated with retardates* 

4* Private Rights (PRRT) 

The higher scores on this subtest indicate that the respondent sees the 
needs of the retarded as overshadowing the private rights of school 
personnel, playground officials, landlords, etc* 

5» Subtle Derogatory Beliefs (SUBT) 

The higher scores on this subtest assume that the respondent is not 
subtly derogatory to the mentally retarded* 

Lower scores assume the respondent to be subtly derogatory towards the 
retarded* 

Scoring 

The MASMR is relatively easy to score. The Scoring Guide lists the point 
value for each response on each item. The List of Items and Subtests gives the 
items that belong to each subtest. To get a subtest score, add the point value 
of each item in a subtest^ For example, the subtest Integration-Segregation 
includes the following items; 1, 6, 11, 16, 21, 26, 31, 36, 4l> 46. To get a 
subtest score for Integration-Segregation, determine (with the Scoring Guide) 
the value of each item in that subtest. The sum of those item values is the 
subtest score* This is repeated for all subtests. 

One can also obtain a Total Attitude Score by summing the subtests. However > 
this score is of less value in terms of specifying the dimensions of a person's 
attitude picture. 



*For a mere cemplete description of the subtests, see Harth, R* Attitudes toward 
minority groups as a construct in assessing attitudes towards the mentally 
O :arded* Education and Training of the Mentally Retarded , 1971, 6, 142-147* 
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tWLE m - PROOftAM MODELS 

1. Name__ , 3, Sex Male ^ Female _ 

^* ll I II ^* School Building and District} 



41 - 50 

51 & over _ Classroom Kumber 



(for mm. teacher only) 

5. Relationship to the EMR class: Parent 

Prlnclfkal 



Supportive Personnel 

(nurse, psychologist, etc.) 
BMR teacher 
Regular Teacher 
Non^Certlfled School Bosployoe 
(Janitor, cook, etc.) 

6. Number of years experience In general education 

In special education 



In present position 



Splnlon^ 'Zr2^'«'io'';/^/" "'^^^^ deferent people. Please give y^ur own 
opinion. There Is no right or wrong answer. 

This booklet contains number statements. Read each statement carefully, 

-If you strongly agree with it, put a check In the column marked "Strongly Agree." 

"rf ^yJji^^if^y P"* « ''^^'^^ i« <^<»l"»n marked "gfeJ." 

-If you mildly disagree with It, put s check In the column marked "disagree." 

" wJagree!""^ ^ P*"' ' marSed 's??ongiy 

Respond to each statement. Put only one check for each statement. - 
How turn the page and go ahead. HORK FASTI 
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Interpretation 



In this eeetlon, there vlll be a dlacussion o£ the meaning of subtest scores* 

tntegratlon-Segregation Overfavorablenesfl 

1 2 

6 7 
11 12 
16 17 
21 22 
26 27 
31 32 
36 37 
41 42 
46 47 



Social Distance 



3 
8 
13 
18 
23 
28 
33 
38 
43 
48 



Subtle Detogatory Beliefs 



Private Rights 

4 
9 

14 
19 
24 
29 
34 
39 
44 
49 



5 

10 
IS 
20 
25 
30 
35 
40 
45 
50 
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In the Bumet of 1972, classes for EMR students In Warren, Ohio at the Market 
High School (tin all EMR school program) were phased out and students were integrat- 
ed in the mdlnstream o£ education and in traditional special education classes* 
Special education and regular education teachers were involved in an intensive 
suntaer vorkshop designed to assist the integration process* 



In December 1972, 300 teachers in the Junior high and senior high buildings 
receiving these EMR students were requested to voluntarily fill out the MASMR. 
One hundred thirty three MASMR^s were returned (44%). Elghteew of these were 
special education teachers j 115 were regular education* Twenty were involved in 
the summer workshop! 113 had not been Involved* Of the twenty who were involved 
In the workshop, six were special education teachers, and fourteen were regular 
education teaehers* 

Within the five sub areas, scores may range from 0 - 40, Six variables were 
recorded per respondent* 

1* Special Ed - Regular Ed 

2, Summer Program - Not Summer 

3* Teaohlng &cperiencet - less than one year - more than 8 

4* Married - Single 

5* Mate - Female 

6« Bachelor Degree - Master Degree 

Individual non correlated t tests were run for each of the groups of scores for 
the five sub tests (5 x 6 « 30 t tests). Running multiple t tests such as this 
increases the probability of significant scores by chance alone. Table I lists 
the sub tests and groups which were significant at the .05 level. 



SIGKlFICAm mm SCORES 



t> .05 

a i>96 

18 Speoldl Education 33*22 G* « 4*28 

SOIS t 2«$4 

115 Rdgular Eudoation 30*88 C - 2«34 



18 Speolal Education 29*17 C • 5*44 

"' PRR¥ jj" K 2 2X 

115 Rdgulat Education 26,15 - 3*76 



20 Summev Program 32*20 (T 2*55 

SDIS t m 1,98 

113 Mot Sunsuer 31*01 C - 1,18 



20 Suimer Program 30.15 ^ > 3*40 

PRRT t - 4.88 

113 Not Summer 25*92 CS* - 3*95 



20 Summer Program 25.75 <r* 4»87 

StJBD t - 2.39 

113 Not Summer 22.95 <sr 3.49 



V6 Bachelor Degree 24.07 »• 3.15 

IJJSB t - 2,11 

51 Maatera Degree 25.84 C - 5.35 

76 Bachelor Degree 21.20 2.09 

OVER t M 3.99 

51 Haetera Dogreo 19,18 ^ « 3.14 



$P«ci$l «dueatl6n tdaehdM scored Bignifio&ntly hlghet than regulat education 
MWhwr oii the social distance factor. Shis Indicates ttiat they do not nilnd 
t|608nlisliia, living near or being associated with retardates more then regtilar 
edt^^atlott tflflchers* Special education teachers also felt that the heeds of the 
ret^rdW overshadowed the private tights of school personnel* landlords, etc, 
m^?* than regular education 

persons Iny^lved In the susner workshop <6 special education, 14 regular education) 
responded the same way as the special education teaehero above* In addition 
ithey were not as subtly derogatory to the nentally retarded as the non workshop 

.pftSClHaelV;"; , -^'M'':^:-: v:;vff 

The statistically slgnlf leant s6ores between the baehelot degree personnel and 
ttaeters degree personnel Indicate that the oasters level personnel were ttore 
in favor of Integration of the retarded and were less willing to attribute over- 
favorable characteristics to the retardate. 

DISCUSSION AM D COMCLyslQt^ 

Ihls Study has severe limitations r^lMtlve to the. statistical ihalysis. If the 
Statistics ^ould be aeeuued to be valid, special educafcloti personnel, appear J:i> 
have ttore accepting attitudes than non special education personnel, Thos^ p^rsdns 
involved In the sunmer workshop have much better attitudes than those not involved* 
Masters degree personnel have more realistic attitude than bachelor degree Per* 
sohnel. Care tmist be noted that th^se conclMslons are stated very cautldtisly and 

effect relationship is Intended* It seems clear, however, that the 
MASMR opinion Inventory Is sensitive enough to measure different attitudes of 
different school personnel* 



OBJECTIVfi lEStlMQ RESULTS 
HOW TO READ TttlS OUTPUT 



1. GRAPHS > 1971-1972 data* 



Thd sample sizes are small* In about 90% o£ the cases i only one or two classes 
took the obJeotive« The following chart shows which objectives have been taken 
by three or more classes. 

OBJECTIVB 



D417J 

C641J 
t}248Z 
02041 

Dim 

D617S 



B937 
C081I 
C463J 
C865j 



b. What to look for - 

There are two graphs and one table for each objective* If the table shows that 
a significant number of students took that objectivo, then the graphs viU be 
valid. It iSi of coursot dangerous to draw conoiusions from the graphs dlone> 
as a particularly "nice" curve could be generated by one or two students. The 
table, then, indicates the nuubet of students who took that objective* The Idft 
, hand graph shows the Tl results. The X's show the percentage of studcrits passing 
Tl while the 0*8 show the percentage of students failing Tl« If no students at 
a mental age level took Tl for that objective, the line should be blank* 

The right hand graph is a bit different. Tlie **s (periods) represeat the 
percentage of students passing both Tl and T2« The XU represent those stvidents 
who passed T2, but either failed Tl or did not take Tl* The 0*8 represent those 
students who failed t2* They may or may not have p^tssed or taken Tl* therefore, 
the periods and X** a together represent the percentage of students who passed f2. 

c* Answers to obvious questions* 

1* Why is the Content Outline Code missing for some of the objectives? 

- Simply because we had no Content Outline Code on record that corresponded 
to that objective* 

V?* .Why do some classes appear to have only one or two students in them? 

B06(i{U8e there was a great deal of bad data for that class* In some 
cases whole classed had to be eliminated** We decided to display the 
bbjectivd anyway I to indicate that it (utd beett used > even though we 
haiirl a^^ata for it* 



KO. OF CUSSES OBJECTIVB OBJECTIVB 
9 D590 

$ C039I C116P 
D531 

5 B951 C640J 

C949P D224I 

4 C031I C041I 

C610J C647J 

D419 D600 
E222 

3 B854P B935 

C032I C037I 

C115P C292P 

C508 C611J 
D251 



3« What Id t:he tesolution o£ the graphs? 



* The rasolution is 1%. As each percentage is calculated it is rounded to 
the nearest even integer between 0 and 100 • So if 1 out o£ 3 passed 
Tl> there would be X*s to 34% on the left side« If 2 out of 3 had 
passed I there would be X's to 667^ Zero percent is left off for readability • 



2« GRAPHS - 1972-1973 data* 
a* Warnings. 

This differs in some respects from the 1971-1972 graphs* There is much less 
missing data* Very few of the IQ's and birthdates are missing* Most of the 
class codes » however^ are missing* This means that the "number of classes^* line 
on the graph is usually inaccurate* It also appears that in 98% of the cases » 
if a student passed Tl, he was not given T2» This means that there are very 
few graphs with periods* For T2» only the students who failed Tl were given T2* 
Thid iihplies that while there are usually larger sample sizes for Tl with the 
1972-1973 datSt the T2 siees are smaller since they represent a subset of Tl* 
Kote that the Content Outline Codes appear on teost of the graphs* 

b* What to look for - 

Mostly the same thing as with the 1971-1972 data* Here» however, the periods 
are few and far between* 

c* Answers to obvious questions* 

1« Why do there appear to be upwards of 30 or 40 students in some classes? 

- Since the class codes were missing in such a high percentage of cases» 
there was no way to tell which class a particular stuU^^nt was In* 
Therefore » all those taking the objective were assumed to be In one 
class unless otherwise specified* 

3* OBJECTIVES LIST - 1971-1972 

This is straightforward and self-explanatory* If used in conjunction with the 
graphs for 1971-1972, the exact classes taking a particular objective can be 
determined* A summary at the end is also given for the userU benefit* A 
"testing*' is defined to be one student taking one objective* The following 
statistics can be easily derived by hand - 

testings ^ 9543 

Average number of objectives taken by each student » # students 983 

- 9.71 

^ testings - 9543 

Average number of students taking each objective » i objectives 595 

« 16*04 

# objectives ^ 595 

Average number of objectives taken by each class « il^ classed Ud 

• 4*02 (comparing to 1* this implies overlap) 

j[ Students 983 » 6*71 
Average number of students in each class « # clasaes 148 ' 

^ testings ^ 9543 -65*16 
Ayetfage number of testings In each class « ^ classes "t?? 



1* 
2* 
3. 




4i OMSCtlWM LIST - l972-m3 

thiti l8 n«»t so sttalghtforwatd because of the missing oUss and student codes* 
About all that can be devlved tttm this Is the number of students <:a\clag each 
objective* In some rata Instances > the class codes Ate given and ate thgn 
useful as they were with the 1971-1972 list. Of the suomaty statistics at the 
end, only the numbet of objectives and the number of testings ate valid* 
Statistic 2 ftom above would bet 

f testings 3922 

Average number of students taking each objective » # objectives - 250 

m 15,69 



9^ 'I. 
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DISTRIBUT ION OF BEH AV IORAL OBJBCtlVES 



Akron Satellite Center 
Ashtabula Satellite Center 
Athens Satellite Center 
Hamilton County Satellite Center 
Louisville Satellite Center 
Mentor Satellite Center 
Toledo Satellite Center 
Tuscarawas Satellite Center 
Dayton 

Ohio State Department of Education 
Jack Showers 
Frank (feller 
Sam Bonham 
Joe 'Todd 
Jacque Cross 
Gerald Ine Parham 
Tom Schled 
Richard Dragln 

Universities 

Bowling Green State 
University of Toledo 
University of Cincinnati 
Kent State University 
Ohio State University 
Ohio University 
Wright State University 
Youngstown University 
Akron University 
Lake Brie College 

Consultants 

Dr. Edward Ueyen 
James McGcttlgan 

Bureau of Education for the Handicapped USOB 
University Microfilms, Ann Arbor, Michigan 
ERIC Clearinghouse 

Instructional Resource Centers 
Edward Stewart, Mllan> Ohio 
Horman Eappln, Dayton, Ohio 
Irma Thomas, Columbus, Ohio 
Linda Bower, Athens, Ohio 
Helen Castle, Ullisboro, Ohio 
Ronald Boley, Gailotti Ohio 
Sheryl Kelson, Lima, Ohio 

Mentor Board of Education 
Dr. Noffsinger 
CoordlnatlDg Center 
Mentor Printing (reference) 



Rir 



Reoaining in inventory 

Total voltunea 
C-l 



Seta 
■55 
31 
25 
.43 
44 
43 
33 
33 
15 



1 
I 
1 
1 
1 
1 
1 
2 



I 
1 
1 
1 
1 
1 
1 
1 
1 
1 



1 
1 

1 
1 
1 



5 
5 
5 
5 
5 
5 
5 

1 
1 

i 
1 

m 



EMR mCHBR PREDICTIOH OP SWDEHt PBRPOmNCB 



these afeudles were conducted to find out how well teachers of educable nentally 
retarded children in Ohio could predict success of their children on specific 
behavioral objectives. It is assumed that prediction of success will have an 
effect on teacher expectancy and thus teacher prediction and teacher expectancy 
are very closely related. (Rosenthal) R. 1966| Thorndike, R* L. 1968} Gosali, 
J. and Meyen» E. 1970.) 

Procedure Itoriber 1 • During tho 1971-72 school year» mote than 1,000 teachers 
of EMR students in Ohio were involved in an ESEA Title HI project, "Frograst 
Models for EMR Students." these teachers represent the entire State In that 
they teach in urban, suburban, rural fara, and rural Appalachian areas in more 
than 50 counties and 200 school districts, these teachers, prinary through 
senior high level, were each involved in a mlnlmua of 15 hours of in-service 
education on Identifying, writing, and field testing behavioral objectives. In 
the field testing process, each teacher chooses objectives froa a bank of ob« 
Jectlves and predicted whether each child In her class would pass or fail the 
objective. After recording this prediction, she actually pre-tested each child 
and recorded their pass or failure perfomance. She was instructed to then 
teach only the students who failed fche pre-test. this prediction * pre-test 
procedure was only part of a total teaching strategy. 

Eighty teachers were randomly selected from those who field tested more than 3,500 
objectifies (20 teachers at each level, primary through «snibr high.) Pour pb- , 
jectlves were randomly selected from each of those submitted by each teacher 
(total 320 objectives, 3,287 prediction - pre-test situations.) 

Procedure Hunger 2 - Because of the pcssibllitles of teachet bias In ths pte-t««; 
situation, a second sanq>le of behavior was collected during the , 1972-73 school 
year. Ten EHR supervisors of teachers in the same geographic areas with similar 
training were requested to do the pre-testing section of this procedure with 10 
of their teachers, therefore, 100 teachers (primary through senior high) pre- 
dicted the performance of each of their students on two objectives chooseh by 
the EMR supervisor (200 objectives, 1,295 prediction - pre-test situations.) 
the supetvisor then pre-tested each child on each objective and recorded the 
performance. 

Over-prediction means that a teacher predicted success but the student failed. 
Under-predlctlon means that a teacher predicted failure but the student passed. 
Correct prediction Included a pass-pass or fail-fail prediction - pre-test 
situation. 

Results! table 1 indicates the percentage of correct, over-prediction and under- 
predlctlon scores for the first study and the second. 

A similar pattern was found even when BMR supervisors did the pre-testing. that 
is teachers were correct most of the time. If they were in error, they over- 
predicted or expected more of their students than was possible. Errors through 
under-predlctlon of student performance was found only 7.4X of the time. Mten 
one combines the correct prediction and over-prediction, EMR teachers in Ohio 
have a realistic expectation or over-expectation about 93% of time* Additionally, 
it may be noted that since there is no significant difference in patterns between 
the first and second study, one could assume that teachers were repotting honestly 
and with little bias during first study. 



Further analysis of the data Indicated that there was no significant difference 
in this pattern for geographical situation (urbani rural and suburban^ subject 
content areas ^ or level of instruction (primary^ intermediatei Junior high, 
and senior high*) 

Conclusions t Based on this representative sample of teachers of EMR students in 
Ohio» the following conclusions can be made: 

1* EMR teachers have a realistic expectation of their student's performance 
about 72% of the time. 

2» When they are in error » they tend to over^predict or expect too much* 

This study was supported by funds made available through ESBA Title III Project 
*45.73-«8-l. 



TABLE 1 

Pewcentages of teacher prediction scores 
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stmRY swtmm by valzmioiis 



The procedures that were followed in the provlelcn of In-servlce teacher 
education (practice) and the systen that was desigtied to effect ehaskgee In 
teacher behevlor are Integral to the innovative character of the project. The 
uee of an ongoing workahop for teachers in widely s^arated geographic areas 
to train then to write and to field test behavioral ^ J actives was the vehicle 
for promoting changes in teacher attitudes and teacher practices. The project 
used ttdsting Regional Special Education Centers as a base for a systems 
i4>proaoh to in-service teacher workshops. 

The <;uality of leadership exemplified by the project director was, in part, 
responsible for the educational climate that enabled the teachers of EHR 
students In the participating geogri^hical areas to identify themselves with 
4 oignif leant progran and to interact with each other* Traditional materiills 
were used to generate measurable behavior objectives which nay be used to 
determine Increments of pupil gain. The combination of innovative character- 
istics of the project can be sunaarlsed as process which includes the develop- 
ment of products* 

Hvaluiftion 

Thie project appears to have been effective in terms of achieving the ob- 
jectives eubmitted for validation. Pre and post testa given to teachers 
participating in the in-service worksbc^s consistently reported significant 
gains (at the .05 level %more). The utillKation of IHC*s (Inftructional 
Haterials Center) as vehicles for change, extention of in-service activities 
to an increasing population of teachers, as well as the development of a 
criterion reference assessment instrtnent all appear to have been achieved. 

Informal conversations with participating teachers (workshop) revealed tiiat 
•ignlfieant behavioral changes also appear to have occurvod in their teaching 
behavior. Little eaq;>irical evidence exists, however, that would substantiate 
this behavior change. 

Perhaps a research design that would focus on the pre and post observations 
of participating versus non participatlgg jeaehers would be useful in the 
documentation of the behavioral changes that i^>pear to have occurred. 

Although the data collective instruments developed by the project appear to 
have face validity, little has been done to establish reliability (used here 
in the statistical aenae.) 

In summary, this project does ^pear to have been effective in achieving its 
Objectives. It is l^ortant to note, however, that the ctianges tiiat seemed 
to have occurred, prtfbably are more related to process variables rather than 
the products produced by the project. 

Colt Effectiveness Analysis 

U D evelopment Costa - An initial grant of $420,000 was followed by a grant 
M $460,000 in the second year to develop a comprehensive set of behavioral 
Objectives for EMR children from primary through high school* Fourteen 
eotttent areel were addressed by approadsuitely 1,006 selected teachers from 
liilli^lililiicticnal resource center areas representing urban, suburban and 
piNii^l iHi^l ais^cti achMi the State* 




c-T 



Thwe teachers were taught to write, teach and evaluate through field testing 
these behavioral objectives. Fifteen volumes each containing 450 behavioral 
objectives for BMR children represent the final product* A special achievement 
Inventory which weasures the EMR students learning of the behavioral objectives 
has also been developed, 

^* Initiation Costs - 'Jhe major thrust of the project was the Involvement on 
a stdte*-wlde basis of certain teachers of man In-servlce education process 
which resulted in the development of 15,000 behavioral objectives. This total 
number was reduced to 6,750 objectives and publishes In fifteen volumes* The 
achievement Inventory will be used In the future to measure pupil learning based 
on mastery of the behavioral objectives. 

Costs for Initiation are based upon establishment of project on a regional basis 
within a state. A budget of $178,700 for one year Is proposed which will cover 
essential staff consisting of a state-wide coordinator, full time secretary, a 
business manager, a regional supervisor for each of eight regions and part time 
secretarial help* Teacher stipends for 25 teachers per region attending four 
sessions each and paid on a basis of $5 per hour cte provided. Haterlal and 
supplies, telephone and allUe are Included also. Not included but also to be 
considered are allowances for travel needs and contractual services for consultant 
services and collection of data base. These Items were not Included because of 
the dependence of cost on the area where the project would be replicated. Also, 
the need for data base and consultant may not be an Initial consideration when 
existing project material Is utilized in the teacher education process. These 
items may be needed In the future. 

3* Operation Cost after Installation - A cost figure of $200,000 Is proposed 
for continuation of the project. This figure provides for Increased salaries 
for staff retained In the project. Teacher Involvement Is continued for further 
refinement of the behavioral objectives and the refinement and use of the special 
pupil achievement Inventory. 

Bxportabilltv 

The exportablllty of the products*' of this project, defined as changes In 
behavior of the teacher-learners and the n:aterlals created In the context of 
the processes used may be viewed as hifth > 

The network of llIC*s was a significant factor In facilitating a state-wide 
project* Other states, without such a netv;ork could apply the processes of this 
project to other regionalized agencies, e.g.. Institutions of higher learning, 
Intermediate units, etc. 

.This project can only be viewed as a state"*wlde activity, generated, supported 
and ultimately put to use in those terms* The LEA served as a vehicle for its 
fiscal and operational aspects. 

High development costs have been spent and states which may appropriatatha Ideas 
flitid materials developed here \rlli not be required. 

The nature of the project permits It be done with existing staff with no spaclal 
equipment or physical facilities. This should make the project attractive to 

;v|§|;;;it|t| ■ tM^W^^^ 

states Interested In this project should not be misled by the project title and 
Mm objectives which directly allude to ''models for Bill dMonM 

8ei^ 



Kftjor Crltttla Batlogs} Suoaary FlodinsB 



nOTEt Take the sub -total from each of the four criterion lt«ma from the 
previous seetlons aod apply It to the spptopH&te scalea below* 
Take the score on vhlch your sub-total rating falls and record it in 
the appropriate colunn to the right. 



a* Inoovativeness 



SCC»B 

Subtotal 
Rating 



5 

X 



10 



5 

Slightly 
Innovative 



10 



15 

1. 



20 



15 

Moderately 
Innovative 



25 

jl 



20 25 
Highly 
Innovative 



b« Effectiveness/Success 



SCORE 

Subtotal 
Rating 



5 

(0-18) 



10 

7I9W 



15 



20 

1 



25 

L 



(55-72) (73-85) 



c. Cost-Effectiveness Analysis/Betonomleal 



SCORE 

Subtotal 
Rating 



5 



(teas 
than 10) 



10 15 20 

Tn^n5 — 05=^55 — ^y4r 



25 

752W 



d. Esportablllty 



SCORE 



5 



Subtotal (1-15) 
Rating 



10 



15 

J. 



20 

JL 



25 

JL 



(16-30) (31-A5) (46-60) (61-75) 



GRAND TOTAL 



Projects vill not be tuwiinated for validation unleas they 
have a nininum of 20 points on eaeh subseore and a niolnua 

Plaaia ptovide a one page typewrittan namtiva statsmaat 

aay areas not addressed in the preceding iiueations« 



